Contribution to the handbook
Priority · Assessed 2026-10-30
The distinction between predictions for additional people, existing participants, existing proposals and new proposals specifies a consequential alternative for estimating preferences. It also raises questions about representation and reliability.
Does this challenge the goal?
In-scope gap
The question investigates a possible preference-estimation mechanism within the existing goal; it does not contest the goal itself.
Whether predicted preferences should play any governing role remains open.
Why this category?
- system specification
- consequential risks and limits
- testing
Predictions are not expressed preferences or consent. The reported comparison beats a specific simple rule, not every alternative, and the study does not establish representation of Brazil’s population or suitability for governing decisions.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Making and interpreting laws →
Authority, accountability and resilience →
Tests, stop conditions and unresolved work →
Read the exploration goalIdentify the kinds of preference data a language model can add, including whose preferences and preferences about which choices.
What we can say so far
A language model can add predicted, rather than directly reported, preferences in several ways described by the authors: - **New people:** predict preferences for people not in the sample, potentially using information such as demographics (Type I). - **Existing people and proposals:** predict additional choices by sampled people among proposals already considered (Type II A). - **Existing people and new proposals:** predict sampled people’s preferences about proposals not previously considered (Type II B). Type II C combines the two Type II approaches. - **New people and new proposals:** predict preferences for both people and proposals outside the original sample (Type III). The choices can be pairwise comparisons of policy proposals. In another system, a personalized reward model predicts each person’s preferences among candidate statements generated by a separate model. These are predictions, not evidence that the people actually expressed those preferences. The authors also suggest that prompts or additional training might elicit preference patterns from training text, but whose views those patterns represent and how faithfully they can be recovered are uncertain.
What remains open
- The supplied evidence does not establish whose preferences are represented in model training text or how faithfully they can be recovered.
- The supplied material does not establish the accuracy or representativeness of all the augmentation types across settings.
Conditions and limits
- The categories describe proposed forms of augmentation, not a guarantee of accurate or representative predictions.
- The policy study aimed to model a specific group of 267 participants, not Brazil’s general population.
- Predictions may change when the order of options is reversed, and groups missing from training data may be poorly represented.
- The source interpretations are not independent verification of the underlying studies.
Objections to consider
- Option-order sensitivity limits confidence in some predicted preferences; the supplied evidence does not resolve it.
- The model comparison documents differing consistency under reversal, without establishing a general remedy.
- The study does not support population-wide claims about Brazil.
- Potentially poor representation of groups missing from training data is not resolved by the supplied evidence.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.