Contribution to the handbook
Priority · Assessed 2026-10-28
Age-related accuracy differences were statistically significant for some models but not others. That is a consequential limitation for evaluating whether a proposed system represents citizens consistently, and supports scrutiny in testing without proving harm in a deployed system.
Does this challenge the goal?
In-scope gap
The question concerns whether model accuracy differs across age groups, an evaluation issue within the existing goal of testing citizen-serving AI. It does not contest that goal or its beneficiary boundary.
The supplied study evidence covers younger-versus-older quartiles, not every age group, and does not establish the size, cause, or wider relevance of the reported differences.
Why this category?
- consequential risk or limitation
- testing
These records derive from one study and shared source material. They do not establish effect size, cause, recurrence, or consequences in civic decisions.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Authority, accountability and resilience →
Tests, stop conditions and unresolved work →
Read the exploration goalState whether differences appeared for every model, identifying models with and without reported significant younger-versus-older differences.
What we can say so far
No. The reported younger-versus-older accuracy differences were statistically significant for Mistral 7B, Falcon 7B, and Gemma 7B; GPT-3.5 Turbo, LLaMA-2 7B, and LLaMA-3 8B were not marked significant at the table’s stated levels. The authors describe the significant differences as a slight tendency to predict younger-quartile participants more accurately.
What remains open
- The supplied excerpts do not give group averages or effect sizes, so the magnitude and practical importance of the differences cannot be assessed.
- The excerpts do not establish whether the patterns would recur in other samples.
Conditions and limits
- This comparison is between participants in the younger and older quartiles, not every possible age grouping.
- The three significant results are marked at the table’s 1% significance level.
- The result and its direction are reported by the authors; figures were not independently interpreted.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
context
Comparing participant groups The authors plan to examine whether models predict choices differently across participants’ self-reported political ideology, age, education, and sex. For these comparisons, they balance the groups by downsampling the group with more participants; the supplied text does not include the results.
Notes that another passage introduces demographic comparisons without reporting their results; it does not provide the model-by-model findings needed here.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
context
Uncertainty around accuracy figures The authors report 95% confidence intervals based on 100 bootstrap iterations for individual-model accuracy, and 99% confidence intervals based on 100 iterations for demographic comparisons.
Provides context about reported confidence intervals, but does not identify which models had significant age-group differences.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
context
How group accuracy was checked To compare prediction accuracy across groups, the authors retrained models on balanced samples when the training data overrepresented one group, then tested accuracy separately for each group.
Describes the group-accuracy comparison method, not the model-specific age results.
Large Language Models (LLMs) as Agents for Augmented Democracy