Contribution to the handbook
Supporting · Assessed 2026-10-28
The held-out testing procedure is useful background for judging what evidence a prediction claim rests on, but this narrow study-method question does not itself specify a governance design or establish civic readiness.
Does this challenge the goal?
None identified
The question asks how one study tested individual-choice predictions; it does not contest the exploration’s goal or boundary.
The supplied accounts describe held-out participant testing, but do not establish generalization to other populations or real civic settings.
Why this category?
- test methodology
- limits of evidence
The cited answers interpret the same underlying study, so their repetition is not independent corroboration.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Tests, stop conditions and unresolved work →
Read the exploration goalNew evidence is being assessed. Any answer below is provisional; parts whose evidence changed or was withdrawn are withheld.
Describe how training and test participants were separated and how predictions were checked against recorded choices.
What we can say so far
The researchers used train-test cross-validation: they trained the models on one group of participants’ choices and held out another group. In the reported split, each group contained 50% of participants. They checked predictions by comparing the models’ predicted pairwise choices for the held-out participants with those participants’ recorded choices, using accuracy and also reporting Kendall’s tau as a comparison measure.
What remains open
No material gap is recorded within this part’s current scope.
Conditions and limits
- This describes the researchers’ test on the study participants; it does not establish prediction performance in other populations or settings.
- The evidence describes Kendall’s tau as a measure but does not provide its results.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Testing individual predictions The researchers test individual prediction by training models on some participants’ choices and checking predictions against choices from participants held out for testing.
Directly describes training on some participants’ choices and checking predictions against choices from participants held out for testing.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
supports
Individual choice predictions The authors report that the models predicted individual policy choices more accurately than a rule assigning people the proposals of the candidate matching their self-reported political orientation.
Confirms use of train-test cross-validation; its comparison with a political-orientation rule is beyond the requested method description.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
supports
Individual prediction results In a test trained on 50% of participants and evaluated on pairwise choices from the other 50%, LLaMA-2 had 76.68% accuracy (±0.0014), GPT 3.5 Turbo 70.4% (±0.0013), Mistral 7B 69.4% (±0.0017), Falcon 7B 69.3% (±0.0014), and BERT SQuAD 50.83% (±0.0015), which the authors describe as not better than random choice.
Specifies a 50%/50% participant split and reports accuracy on pairwise choices from the held-out group.
Large Language Models (LLMs) as Agents for Augmented Democracy