Contribution to the handbook
Priority · Assessed 2026-10-28
The evidence describes comparisons across sample sizes and reports that augmentation improved some matches to the study’s full participant aggregate, with results varying by sample size. This is consequential to assessing the limits and potential dependencies of using model predictions when participation data are sparse. It does not show that predictions represent people outside the study or are suitable for real decisions.
Does this challenge the goal?
In-scope gap
The question concerns how sample size affects preference-model results, a relevant dependency and testing issue for proposed representation methods.
The evidence is confined to one study’s participants and measures, and some reported sample-size results are not available in the packet.
Why this category?
- Identifies a consequential data-coverage dependency for a proposed representation approach.
- Provides a concrete testing comparison and a limited reported result.
- Preserves the distinction between matching the study sample and representing a wider public.
Reported improvements are limited to the study’s data and measures; the supplied evidence does not establish general performance or population representativeness.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Authority, accountability and resilience →
Tests, stop conditions and unresolved work →
Read the exploration goalState whether they examined different training-sample sizes and describe the comparison they made.
What we can say so far
Yes. The researchers examined multiple training-sample sizes. They compared results across random samples of participants, including 5%, 25%, and 50% samples; they also reported models trained on 5%, 25%, and 75% samples alongside the 50% main sample. The reported comparisons included model accuracy and, in a Type I B test, how augmenting samples with model predictions affected aggregate-preference estimates. The supplied evidence reports gains for the 5% and 25% augmentation tests, but does not provide separate numerical results for the 50% test.
What remains open
- The supplied text does not provide the accuracy values from the figure comparing 5%, 25%, and 50% samples.
- The supplied text gives no separate numerical augmentation result for the 50% sample.
Conditions and limits
- These are descriptions of analyses in the study, not evidence that the results generalize to other settings.
- The reported augmentation gains are specific to the study's data and test design.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Different training-sample sizes The authors also report results for models trained on random samples of 5%, 25%, and 75% of participants, in addition to the 50% training sample used for the main individual-prediction test.
States that results were reported for models trained on 5%, 25%, and 75% samples in addition to the 50% main sample.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
supports
Predictions improved some sample matches In the Type I B test, the authors fine-tuned six models on the same preference data: GPT-3.5 Turbo, LLaMA-2 7B, Falcon 7B, Mistral 7B, LLaMA-3 8B and Gemma 7B. They tested initial random samples of 5%, 25% and 50%, adding predictions for 20% of the people who remained outside each sample. For the 5% sample, R² rose from about 30% to about 75%; for the 25% sample, the gain was about 7 to 10 percentage points. The supplied text gives no separate numerical result for the 50% sample.