Contribution to the handbook
Priority · Assessed 2026-10-28
Representation accuracy and who may be poorly served are consequential limits on using models to inform citizen preferences. The packet includes reported accuracy differences by political view and education, and separately identifies unmeasured risks from missing training coverage, uneven data access, and digital barriers; distinguishing these is important to safeguards and evaluation (q-live-75fb40c502c3a5f2b905fcaa; s-hidalgo-augmented-democracy-dde7ed7ec2a90dac; s-hidalgo-augmented-democracy-7c263642bd1577ae; s-hidalgo-augmented-democracy-02cfac3fc0a9495f; s-landemore-inclusive-democracy-ed0922d39d52bc60).
Does this challenge the goal?
In-scope gap
The question probes whether models represent different citizens well, including those outside tested categories. That is an empirical gap within the citizen-serving goal, not a claim that the goal or beneficiary boundary should change.
The packet identifies some subgroup accuracy differences and possible coverage concerns, but does not establish poor fidelity for the proposed populations beyond those comparisons (q-live-75fb40c502c3a5f2b905fcaa; s-hidalgo-augmented-democracy-7c263642bd1577ae; s-hidalgo-augmented-democracy-dde7ed7ec2a90dac; s-hidalgo-augmented-democracy-02cfac3fc0a9495f).
Why this category?
- Examines a material risk to whose preferences models may represent accurately.
- Separates measured subgroup comparisons from proposed risks and untested populations.
- Supports safeguards and tests without claiming that unmeasured groups have already been harmed.
The packet reports particular models, groups, participants, and languages; it does not establish fidelity for every population, intersection, or setting. Some cited concerns are proposals or risks, not measured model errors.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
- Which citizens might a model represent poorly?
- Which citizens might a model represent poorly, especially among populations not covered by the tested demographic and language dimensions, and what evidence identifies them?
- Unequal data access is a concern
- Results changed little in tested variations
- Comparing participant groups
- The study’s scope
- Some groups may be missing
- Findings on subgroup predictions
- Differences by political view and education
- Accuracy gaps raise a representation concern
- How group accuracy was checked
- Some people could be excluded
- Representing intersecting identities
Authority, accountability and resilience →
Tests, stop conditions and unresolved work →
Read the exploration goalDetermine whether any reported subgroup-accuracy comparison directly establishes poorer fidelity for a population outside the specified tests; identify the population, model, and setting if so, without extrapolating beyond them.
What we can say so far
No qualifying comparison is identified in the supplied evidence. The reported lower-accuracy comparison concerns conservative and centrist participants and non-college-educated participants across six models. Political ideology and education are among the demographic dimensions listed as tested, so this finding concerns groups within those dimensions, not a population outside them. The separate gender comparison attributed to Argyle and colleagues reports women as about one percentage point more accurately predicted; gender is also a listed tested dimension. Neither comparison establishes poorer fidelity for an out-of-dimension population.
What remains open
- The supplied evidence identifies no population outside the specified demographic dimensions with directly measured poorer fidelity; it therefore provides no such population, model, and setting to report.
Conditions and limits
- This conclusion is limited to the comparisons described in the supplied evidence; it does not establish that no such comparison exists elsewhere.
- The lower-accuracy result is reported across six models and the study participants, but the supplied passage does not name the models or provide further setting details.
- The Argyle comparison is attributed to an earlier study, not presented as a new result from the publication discussed.
Objections to consider
- This is a material reported accuracy difference, but its lower-accuracy groups are within the specified ideology and education dimensions; it does not establish poorer fidelity for a population outside those tests.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Results changed little in tested variations The authors report that accuracy did not depend strongly on which demographic details were used. The tested categories were political ideology (liberal, conservative or centrist), location (city and state), sex (male or female) and education (non-college educated or college educated). The demographic tests used GPT-3.5 Turbo and Mistral 7B. They also report little change when prompts and consultation switched from English to Portuguese.
Lists the tested demographic dimensions, including ideology, education, and sex, and distinguishes those tests from the English-to-Portuguese variation.
Identify populations or views named as possible concerns because of missing training-data coverage, intersecting identities, unequal data access, or barriers to digital participation; distinguish each stated concern from evidence of model error.
What we can say so far
The authors identify people or political views from groups missing from training data as a possible representation concern. They also connect accuracy differences across participant groups to the broader concern that language models may represent some parts of the public better than others. The supplied evidence does not name specific populations, intersecting identities, groups facing unequal data access, or people facing barriers to digital participation. These are stated concerns about possible fidelity, not evidence that the models made errors for those groups or actually overrepresented anyone.
What remains open
- No specific populations or views beyond the general category of groups missing from training data are named.
- No evidence is supplied about intersecting identities, unequal data access, or barriers to digital participation.
- The evidence does not establish that any named or unnamed group experienced model errors.
Conditions and limits
- These are the authors’ stated concerns and implications, not independently verified findings.
- The reported accuracy differences are not specified here by participant group.
- The source does not measure real-world overrepresentation or identify which groups are affected or the size of any difference.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Accuracy gaps raise a representation concern The authors connect the accuracy gaps across participant groups to the wider question of whether language models may represent some parts of the public more than others.
Reports the authors’ link between accuracy differences across participant groups and a broader concern about whose views may be represented well, while noting that real-world overrepresentation was not measured.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →
State which proposed populations have no direct fidelity evidence in the supplied material and clarify how the tested categories, participants, models, and languages limit any claim about them.
What we can say so far
The supplied material identifies people whose views are missing from model training data as a proposed risk: models may represent them less well. It does not name specific such populations or provide direct fidelity measurements for them. Nor does it provide direct fidelity evidence for people defined by demographic characteristics beyond the reported comparisons, or for languages beyond English and Portuguese. Age is named in a planned subgroup comparison, but the supplied passage gives no age-comparison results. A male/female input variation test is not itself evidence that predictions are equally accurate for male and female participants; a separate, earlier study by Argyle and colleagues reportedly found models about one percentage point more accurate at predicting women’s preferences, but that finding is not a result for the models and participants in the study discussed here. The evidence distinguishes input variations from subgroup accuracy. For input variations, the authors report little change when using specified ideology, city/state, sex, and education details with GPT-3.5 Turbo and Mistral 7B, and when switching consultation from English to Portuguese. Separately, across six models, they report higher accuracy for liberal participants than conservative and centrist participants, and for college-educated participants than non-college-educated participants. Those subgroup findings are bounded to the study’s participants and models; the study models preferences for 267 participants and does not estimate Brazil’s general population. The supplied material therefore supports concern and some specific measured differences, but does not establish which other populations are poorly represented or how broadly any result generalizes.
What remains open
- No specific populations missing from training data are identified, and no direct fidelity measurements are supplied for those populations.
- No age-comparison results are supplied; the material also does not establish fidelity for untested demographic categories or languages beyond English and Portuguese.
- The material does not establish causes of the reported accuracy differences or whether they persist in other populations, models, questions, or settings.
Conditions and limits
- The input-variation results concern the named categories, GPT-3.5 Turbo and Mistral 7B, and English-to-Portuguese changes; they do not establish equal fidelity across groups or languages generally.
- The reported ideology and education accuracy differences concern the participants and six models in the study, not all people or models.
- The 267 participants are the study’s target group; the material says they do not support estimates for Brazil’s general population.