Contribution to the handbook
Priority · Assessed 2026-10-28
The comparison provides a concrete test of whether models predict individual choices better than a simple alternative, and the evidence reports both broad results and exceptions. That makes it relevant to evaluating a proposed preference-estimation mechanism. It does not show that model performance is sufficient for representing citizens or for making governance decisions.
Does this challenge the goal?
In-scope gap
The question examines a possible test of preference predictions relevant to serving citizens; it does not contest the goal or prescribe how citizens’ interests must be identified.
The reported comparison is limited to the study’s participants, proposals, models and specific party-based rule.
Why this category?
- meaningful alternatives
- testing
- consequential limits
The result is from one study and one benchmark; it does not show generalization, adequate accuracy for governance, or that models captured the reasons behind participants’ choices.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Making and interpreting laws →
Authority, accountability and resilience →
Tests, stop conditions and unresolved work →
Read the exploration goalCompare the models’ prediction accuracy with the simple party-based rule for the task tested.
What we can say so far
Yes. The authors report that the models were more accurate than the simple party-based rule in 21 of 24 comparisons. The reported exceptions involved liberals choosing proposals from Bolsonaro’s program for Mistral, Falcon, and Gemma. In one reported comparison, LLaMA-2 7B was 15 percentage points more accurate than the rule, and all six models reached at least 80% accuracy versus 71% for the rule. The benchmark assigned proposals according to participants’ self-reported political orientation. These results describe the study’s models, participants, proposals, and comparison methods; they do not establish performance beyond those settings.
What remains open
- The supplied evidence does not quantify how option-order sensitivity affected the reported model-versus-rule accuracy comparisons.
- The comparison does not establish that the models outperform other possible prediction rules.
Conditions and limits
- This is an author-reported result from the studied models, participants, proposals, and comparison methods; it does not establish performance in other populations or settings.
- The party-based rule used self-reported political orientation to assign proposals associated with Lula or Bolsonaro.
- The authors also report prediction sensitivity to option order in some cases; the supplied evidence does not establish how that sensitivity affected the accuracy comparisons.
Objections to consider
- Reported option-order sensitivity may affect the robustness of model predictions, but the supplied evidence does not show how it affected the accuracy comparisons with the party rule.
- The reported differences in option-order consistency qualify interpretation of the accuracy results; the evidence does not quantify their effect on the direct comparison with the party-based rule.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Models beat a simple party rule The models also predicted group preferences more accurately than a rule that assumes people always choose the policies of their preferred party or politician.
Reports that the models predicted group preferences more accurately than a rule based on participants’ preferred party or politician.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source ↗