Contribution to the handbook
Supporting · Assessed 2026-10-29
The study’s model and prompt coverage helps readers judge how far its preference-prediction findings can be relied on. This is useful evidence context rather than a standalone governance-system design.
Does this challenge the goal?
In-scope gap
The question examines the breadth and omissions of a study’s testing, not the governance goal itself.
It does not establish whether omitted tests would change the findings.
Why this category?
- Evidence limits and testing context
The records are interpretations of one shared source, not independent confirmations. Wider model or prompt testing could produce different results, but that is not established.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Tests, stop conditions and unresolved work →
Read the exploration goalDescribe the breadth and stated omissions of both model testing and prompt testing, without assuming omitted tests would change results.
What we can say so far
Testing was substantial but not exhaustive. The authors fine-tuned six language models—LLaMA-2 7B, LLaMA-3 8B, GPT-3.5 Turbo, Mistral 7B, Falcon 7B and Gemma 7B—and included BERT SQuAD as a simpler benchmark. They tested individual and aggregate preference predictions, including different training-sample sizes. GPT-4 and Claude 3 were excluded because fine-tuning options were not publicly available to the authors at the time; multi-agent systems were not tested. Prompt-related checks included reversing the order of policy options, varying demographic inputs for GPT-3.5 Turbo and Mistral, and switching prompts and consultation from English to Portuguese. The authors reported little change in the latter two tests, but also found order inconsistency in some cases. They acknowledged that they had not tested many variations of the prompts used for training and querying. They also reported temperature sensitivity checks, though the supplied evidence does not specify the full range of settings. These were tests on a specific set of Brazilian election-policy preferences from 267 participants, not an exhaustive test across models, prompts or populations. The paper does not establish whether the omitted tests would have changed the findings.
What remains open
- The effect of untested models, prompt variations and multi-agent systems on the findings is unknown.
- The supplied material does not specify the full range of temperature settings or provide all model-by-model order-consistency rates.
Conditions and limits
- The account reports the authors’ study and its stated limitations, based on interpretations of a shared preprint rather than independent verification.
- Demographic-input tests used GPT-3.5 Turbo and Mistral 7B; the language and order checks do not establish robustness to all prompt variations.
- The study’s preference data and validation concern the 267 participants and the specific 2022 Brazilian election setting.
Objections to consider
- Many prompts and some newer models were not tested; their possible effect remains unknown.
- Multi-agent systems were not tested, so their performance in this setting remains unknown.
- Testing was confined to the study group and cannot establish population-wide performance.
- Reversing option order was tested, but some predictions remained inconsistent; the full frequency is not supplied.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
context
The volunteer policy data The study uses volunteer choices between randomly paired policies from the two leading candidates’ 2022 Brazilian presidential programs. The choices came from an experiment involving collaborative construction of a government program.
Describes the policy-choice data.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
supports
Six fine-tuned models The authors fine-tune six existing language models on the volunteer data: Llama-2 7B, LLaMA-3 8B, ChatGPT 3.5 Turbo, Mistral 7B, Falcon 7B, and Gemma 7B.
Names the six fine-tuned models.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
context
Individual choice predictions The authors report that the models predicted individual policy choices more accurately than a rule assigning people the proposals of the candidate matching their self-reported political orientation.
Describes cross-validation and a simple individual-prediction benchmark.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗