A starting point for discussion
The idea
In the Type I B test, the authors adjusted six models using the same preference data: GPT-3.5 Turbo, LLaMA-2 7B, Falcon 7B, Mistral 7B, LLaMA-3 8B, and Gemma 7B. They tested initial random samples of 5%, 25%, and 50%, adding predictions for 20% of the people who remained outside each sample. For the 5% sample, R², a score for how closely two patterns match, rose from about 30% to about 75%; for the 25% sample, the gain was about 7 to 10 percentage points. The supplied text gives no separate numerical result for the 50% sample.
Place in the reasoning
AI assessment · Connected to reasoning
This reports improved fit to the full study sample for specified augmentation tests; the results remain tied to the study data and reported configurations.
Consistent study results may not reflect the wider public
The reported estimates concern the study sample’s aggregate preferences, not a broader population.
Different prediction checks reveal different limits
It reports specific augmentation performance while explicitly limiting generalization beyond the study setting.
What this means and why it matters
What this means
The authors compared random samples with samples expanded using predictions about additional people. They report the largest stated gain for the 5% sample and a smaller stated gain for the 25% sample; 50% was also tested, but the supplied text gives no separate result for it.
Why it matters
The stated results suggest that predictions helped reproduce the preferences recorded for all study participants under these test conditions. They do not establish that the predictions reflect the preferences of people outside the study.
Conditions & assumptions
Source coverage: The full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. These figures describe tests on the study’s data, with models adjusted using the same data and predictions added for a stated share of people outside the initial sample. The text identifies 50% as a tested sample size but gives no separate numerical result for it. These tests do not show how well the method would work in other settings. Addresses: The reported result for adding Type I B predictions
Questions this connects to
AI-suggested relationships. Authors have not necessarily responded to one another.