A starting point for discussion
The idea
To summarize policy preferences in a group, the authors define each proposal’s win rate as the share of recorded choices between two proposals in which it was selected. They then compare win rates from a sample with those from all participants in the study.
Place in the reasoning
AI assessment · Connected to reasoning
Defines a study-specific way to summarize and compare group preferences, while explicitly limiting what such a match represents.
Consistent study results may not reflect the wider public
The metric compares a sample with the full study sample and therefore does not establish preferences of a wider public.
Different prediction checks reveal different limits
Specifies the win-rate measure used in group-level prediction comparisons; a match on this measure is not a complete reliability test.
What this means and why it matters
What this means
A proposal’s win rate is the number of recorded choices in its favor divided by the number of recorded comparisons involving it. The authors compare these rates for all participants in the study and for a random sample.
Why it matters
This measure lets the authors examine whether a smaller sample can reproduce the study participants’ relative levels of support for proposals.
Conditions & assumptions
Source coverage: The full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. This approach summarizes choices even when participants did not compare every possible pair of proposals. Its conclusions concern how well a sample reproduces the results for all participants in the study, not necessarily the preferences of a wider public.
Questions this connects to
AI-suggested relationships. Authors have not necessarily responded to one another.