Contribution to the handbook
Supporting · Assessed 2026-10-28
The question supplies technical background on the models and their use. The evidence describes fine-tuning and held-out testing, while also reporting option-order sensitivity and limits on generalizing the results. These details can inform tests, but do not establish a governance capability or implementation choice.
Does this challenge the goal?
In-scope gap
Understanding model adaptation and evaluation is within scope as background for testing possible AI-assisted preference representation; the question does not contest the goal.
The evidence concerns a specific study and does not show that the models would perform similarly in other settings or governance functions.
Why this category?
- useful background
- testing
- consequential limitations
The packet gives study-specific methods and reported limitations, not evidence of performance in a city, country, or real civic decision process.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Tests, stop conditions and unresolved work →
Authority, accountability and resilience →
Read the exploration goalNew evidence is being assessed. Any answer below is provisional; parts whose evidence changed or was withdrawn are withheld.
Identify the tested models and explain how the authors adapted and used them with the study data.
What we can say so far
The authors fine-tuned six existing language models on preference data collected during Brazil’s 2022 presidential election: LLaMA-2 7B, LLaMA-3 8B, GPT-3.5 Turbo, Mistral 7B, Falcon 7B, and Gemma 7B. They also included BERT SQuAD as a simpler benchmark; the supplied evidence does not describe it as one of the six fine-tuned models. The study used paired policy-proposal choices to build preference representations and predictions. Prompts included participants’ recorded choices and selected characteristics (such as age, ideology, location, education, and sex). The researchers included both orders of each proposal pair and used LoRA to adapt about 12% of model parameters, leaving the rest fixed. They evaluated individual-choice predictions against held-out participants’ choices and also used predictions in sample-augmentation tests, comparing proposal win rates in augmented samples with those in the full participant data. The reported methods and tests concern this study’s data; they do not establish performance in other populations or settings.
What remains open
- The supplied evidence does not show that reversing option order eliminates inconsistency; the reported order-sensitivity limitation remains unresolved.
Conditions and limits
- This is a description of the authors’ reported methods, based on interpretations of the same preprint, not independent corroboration.
- The tests use the study’s collected preference data and do not establish transfer to other populations, elections, or settings.
- Including both option orders was an attempted response to order sensitivity; the supplied evidence does not show that it resolved the reported inconsistency.
Objections to consider
- The reported order sensitivity is material to how predictions were used. The evidence that both orders were included describes an attempted mitigation, not evidence that the reported inconsistency was eliminated.
- The reported differences in consistency across models qualify the prediction method; the supplied evidence gives no exact model-by-model rates or evidence that the issue was resolved.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Six fine-tuned models The authors fine-tune six existing language models on the volunteer data: Llama-2 7B, LLaMA-3 8B, ChatGPT 3.5 Turbo, Mistral 7B, Falcon 7B, and Gemma 7B.
Names the six fine-tuned models and says they were adapted using volunteer preference data to predict choices and group preferences.