Contribution to the handbook
Supporting · Assessed 2026-10-28
The evidence supplies concrete modeling and evaluation details, including order sensitivity, group-accuracy differences, and limits on prompt and model coverage. These help frame scrutiny of preference-prediction methods, but the results are study-specific and do not establish that such models can safely represent citizens.
Does this challenge the goal?
In-scope gap
It asks about a specific model’s training and prediction setup, relevant to examining technical limits and tests within the existing goal, without contesting that goal.
The study is a bounded prediction experiment, not a test of an AI governance system.
Why this category?
- implementation
- risks
- tests
The supplied material includes a reported temperature robustness result and separate, result-free temperature-test descriptions; it also omits many exact results and does not establish generalization beyond the study.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
- How did the researchers train Falcon 7B Instruct and set up model predictions?
- Did the researchers check whether temperature affected predictions?
- Did a training setting appear to change the reported results?
- How complete was the model and prompt testing?
- Which models were compared, and what distinguishes their training backgrounds?
- How did the researchers adapt models to predict participants’ choices?
- People were represented with limited data
- Models in the comparison
- Why the experiment used pairs
- A sparse set of comparisons
- Testing individual predictions
- Adapting the models
- Choices and option order
- Six fine-tuned models
- Accuracy across random samples
- Portuguese prompt example
- Not all models or prompts were tested
- Temperature tests are reported
- Temperature setting
- Participant details used as context
- Keeping model architecture consistent
- Anonymous pairwise comparisons
- Changing option order could change answers
- Two approaches did not improve results
- Where the data came from
- Six models tested on collected preferences
- Participants compared policy proposals
- The volunteer policy data
- Temperature checks
- Individual prediction results
Authority, accountability and resilience →
Tests, stop conditions and unresolved work →
Read the exploration goalNew evidence is being assessed. Any answer below is provisional; parts whose evidence changed or was withdrawn are withheld.
Describe the checkpoint, adaptation and training settings, and how prediction temperature was set.
What we can say so far
The authors list Falcon 7B among the models fine-tuned on volunteers’ paired policy choices, but the supplied evidence does not identify the exact Falcon 7B Instruct checkpoint. For the models generally, they describe prompts combining participant details with recorded choices, including both orders of each proposal pair, and LoRA adaptation of about 12% of parameters while the remaining weights stayed fixed. They say they used temperature zero during training for reproducibility. The supplied evidence does not establish that these settings were identical for Falcon specifically or state the temperature used when generating its predictions.
What remains open
- The exact Falcon 7B Instruct checkpoint is not identified in the supplied evidence.
- Falcon-specific training settings are not established beyond the study-wide method description.
- The temperature used to generate Falcon’s predictions is not specified.
Conditions and limits
- The prompt and LoRA details are reported for the researchers’ model-adaptation method generally, not established here as a Falcon-specific configuration.
- The reported temperature-zero setting concerns training; it must not be treated as the prediction-temperature setting.
- This is an attributed account of the researchers’ method, not independent verification.
Objections to consider
- Reversing proposal order was part of the setup, but reported order sensitivity means it cannot be described as fully corrected. The answer makes no such claim.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
context
The volunteer policy data The study uses volunteer choices between randomly paired policies from the two leading candidates’ 2022 Brazilian presidential programs. The choices came from an experiment involving collaborative construction of a government program.
Identifies the volunteer policy-choice data used in the study.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗