How might language models produce preference estimates?

Contribution to the handbook

Priority · Assessed 2026-10-28

This bears directly on a possible mechanism for identifying citizen preferences and on whether its outputs could be relied on. The evidence also identifies material limits: predictions varied by group and option order, and performance on study participants does not establish representativeness beyond them. These issues make the question relevant to system safeguards and testing, without establishing that the method should be adopted.

Does this challenge the goal?

In-scope gap

The question investigates methods for estimating preferences within the existing goal of understanding citizens’ interests. It does not contest that goal or claim that model estimates are accurate.

The evidence describes proposed elicitation methods and study-specific results, but does not establish that models can faithfully recover the preferences of a wider public.

Making and interpreting laws →

Authority, accountability and resilience →

Tests, stop conditions and unresolved work →

Explain how prompts or additional training could elicit preference patterns from model training and turn them into estimates, while distinguishing the possibility from demonstrated accuracy.

What we can say so far

The authors propose that prompts or additional training might draw out preference patterns represented in a model’s training text. In practice, prompting can provide a person’s characteristics and recorded choices as context; additional training can adapt a model on preference data, after which it can predict choices on proposals or people not directly represented in the input. Predictions can then be combined into estimates of group preferences. The study’s fine-tuning and prediction setup illustrates one such approach, but it trains on collected volunteer preference data; it does not establish that preferences can be faithfully recovered from a model’s original training text. The authors’ proposed elicitation is therefore a possibility, not demonstrated accuracy. The reported cross-validation results concern the study’s particular participants and models, not general accuracy at recovering public preferences.

What remains open

• The evidence does not explain in technical detail how prompts or training would identify and extract preference patterns from general training text.
• It does not establish whose preferences are represented in training text or how faithfully those preferences can be recovered.
• The evidence does not establish accuracy or generalizability beyond the particular study setting.