Contribution to the handbook
Supporting · Assessed 2026-10-28
The question and evidence offer a concrete example of measuring pairwise policy preferences and comparing sample estimates with the study’s full sample. The stated validation limits make it useful background for evaluating preference-aggregation methods, but it is not a demonstrated governance design.
Does this challenge the goal?
In-scope gap
The question concerns how group preferences are measured and compared, a detail relevant to exploring how citizens’ interests could be identified and evaluated. It does not contest the goal itself.
The evidence describes a study-specific measure and comparison; it does not establish that the measure is suitable for broader publics or governance decisions.
Why this category?
- Provides a specific preference measure and comparison approach relevant to evaluation
- Distinguishes matching the study sample from representing a wider public
The supplied records derive from the same source. The measure concerns the study participants and does not by itself establish accurate representation of people outside the study.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
- How did the authors measure and compare group-wide policy preferences?
- Testing whether a model improves a sample
- Testing group-level preferences
- The data used in the comparison
- Anonymous pairwise comparisons
- Summarizing group preferences with win rates
- The volunteer policy data
- Estimates of group preferences
- Comparing sample and full-sample patterns
- Measures of agreement
Making and interpreting laws →
Tests, stop conditions and unresolved work →
Read the exploration goalNew evidence is being assessed. Any answer below is provisional; parts whose evidence changed or was withdrawn are withheld.
Define the proposal-level measure the authors used to summarize group-wide preferences.
What we can say so far
The authors summarized each proposal’s group-wide preference by its win rate: the share of recorded pairwise choices in which that proposal was selected.
What remains open
No material gap is recorded within this part’s current scope.
Conditions and limits
- The win rate is based on recorded pairwise choices; the source says this can summarize choices even when participants did not compare every possible pair.
- Comparisons of sample and full-study win rates concern reproduction of the full study sample, not necessarily preferences in the wider public.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Summarizing group preferences with win rates To estimate group-wide policy preferences, the authors define each proposal’s win rate as the share of recorded pairwise choices in which it was selected, then compare win rates from a sample with those from the full study sample.
Defines the proposal-level win-rate measure.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
context
Anonymous pairwise comparisons Participants compared two policy proposals at a time on an online platform, and their participation was anonymous.
Describes the anonymous pairwise-comparison method underlying the recorded choices.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
Describe how they compared that measure across a sample and the full study sample.
What we can say so far
They summarized group-wide preferences with each proposal’s win rate: the share of recorded pairwise choices in which it was selected. They compared the sample’s proposal win rates with the full study sample’s win rates using the squared Pearson correlation; a score of 100% represents identical win-rate patterns. The comparison concerned reproducing the study participants’ aggregate preferences, including comparisons of ordinary and model-augmented samples with the full group.
What remains open
No material gap is recorded within this part’s current scope.
Conditions and limits
- The comparison concerns the study sample, not necessarily preferences in a wider public.
- A match in proposal win-rate patterns does not by itself establish a match in individual choices or other features of the group.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Summarizing group preferences with win rates To estimate group-wide policy preferences, the authors define each proposal’s win rate as the share of recorded pairwise choices in which it was selected, then compare win rates from a sample with those from the full study sample.
Defines proposal win rates and explicitly describes comparing sample and full-study-sample win rates.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
context
Measures of agreement The authors use accuracy and report Kendall’s tau as another way to compare choices. They say F1 is not a distinct measure for these data because each correct choice creates both a true positive and a true negative, while each incorrect choice creates both a false negative and a false positive; as a result, precision, recall, F1, and accuracy are equal.
Reports accuracy and Kendall’s tau as additional agreement measures; these are not needed to describe the win-rate comparison.