Contribution to the handbook
Supporting · Assessed 2026-10-28
The reported confidence intervals, bootstrap procedure, and group-comparison tests provide useful methodological context for interpreting model-accuracy results and designing evaluation. The supplied evidence also notes that the procedures do not reveal effect size, direction, or practical importance on their own (s-hidalgo-augmented-democracy-52b6046d66a4eef5; s-hidalgo-augmented-democracy-617ef55b1c29568f).
Does this challenge the goal?
None identified
The question concerns how reported accuracy was quantified and compared, which is an in-scope testing detail rather than a challenge to the exploration’s goal.
Reported statistical procedures do not by themselves establish that the measures are adequate for a real governance system.
Why this category?
- Useful testing-method detail for interpreting accuracy claims; no direct system specification or demonstrated operational consequence.
The packet does not provide further calculation details, effect sizes, or information sufficient to judge the practical significance of the tests.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Tests, stop conditions and unresolved work →
Read the exploration goalNew evidence is being assessed. Any answer below is provisional; parts whose evidence changed or was withdrawn are withheld.
Identify the uncertainty measures and procedures the authors report for the accuracy figures, distinguishing figures where the procedures differ.
What we can say so far
The authors report 95% confidence intervals based on 100 bootstrap iterations for individual-model accuracy, and 99% confidence intervals based on 100 iterations for demographic comparisons. Separately, for comparing average accuracy between participant groups, they report two-sample t-tests allowing unequal variances, using models trained on half of the participant data. These tests are a group-comparison procedure, distinct from the confidence intervals reported for the figures.
What remains open
- Further details of the confidence-interval calculation are not provided in the supplied evidence.
- Group sizes, accuracy averages, and effect sizes are not provided in the supplied evidence.
Conditions and limits
- The confidence-interval procedures are specified in figure captions; the figures were not independently interpreted.
- The authors’ description provides no further details about the uncertainty calculation.
- The t-tests test group-average differences but do not by themselves convey the size or practical importance of a difference.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Uncertainty around accuracy figures The authors report 95% confidence intervals based on 100 bootstrap iterations for individual-model accuracy, and 99% confidence intervals based on 100 iterations for demographic comparisons.
Reports the distinct confidence-interval levels and iteration counts for individual-model accuracy and demographic comparisons.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
supports
Testing group accuracy To compare average accuracy between participant groups, the researchers used two-sample t-tests that allow the groups to have unequal variances. The tests used models trained on half of the participant data.