Contribution to the handbook
Priority · Assessed 2026-10-28
Whether people can meaningfully understand model choices is consequential for scrutiny of a proposed preference mechanism. The study authors report no adequate way to explain why a model selects one proposal over another, while the other supplied source distinguishes interpretable from non-interpretable systems without specifying what level of explanation enables effective checking. This is a material limit, not proof that every model choice is impossible to explain.
Does this challenge the goal?
In-scope gap
The question asks about explaining model outputs, a relevant safeguard and accountability issue within the existing goal; it does not contest the goal or its beneficiary boundary.
The evidence identifies a limitation but does not test what explanations citizens would find adequate or whether explanations improve oversight.
Why this category?
- Consequential transparency and scrutiny limitation
- Open question about adequacy of explanations
The three records do not compare explanation methods or establish a standard for adequate explanation. The cited study’s limitation is not independent evidence that explanations cannot be developed.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Authority, accountability and resilience →
Tests, stop conditions and unresolved work →
Read the exploration goalAssess how well a model’s preference between proposals can be explained to people, distinguishing an account of its output from a reliable explanation of why it chose that proposal.
What we can say so far
The Hidalgo authors state that their approach has no good way to explain why a language model favors one policy proposal over another. This supports a limited conclusion that, in their account, the model’s proposal preference is not adequately explained. It does not establish that an explanation of the output would reliably reveal why the model chose it: the supplied material neither tests explanation methods nor assesses whether any account faithfully identifies the choice’s reasons. Danaher’s distinction between systems people can explain in ordinary language and systems too complex for them to understand provides general context, not a finding about this model’s proposal choices.
What remains open
- No evidence evaluates whether an account of a proposal preference faithfully explains why the model selected it, rather than merely describing its output.
- No explanation methods are compared, and the material does not establish what level of explanation would be adequate for checking the model’s reasons.
Conditions and limits
- The Hidalgo claims are reports of the authors’ position in the retrieved preprint, not independently verified empirical findings.
- The supplied Hidalgo material does not propose or test a way to address the explanation limitation.
- Danaher’s general distinction comes from an accessible author-manuscript mirror, not the publisher typeset version.
Objections to consider
- The passage indicates that some systems can be explained in ordinary language, so the Hidalgo limitation should not be generalized to every model. The answer confines its conclusion to the authors’ account of their approach and treats Danaher’s distinction only as general context.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
supports
Reasons for choices are unclear The authors say there is not yet a good way to explain why a language model chooses one policy proposal over another.
Reports the authors’ specific claim that there is not yet a good way to explain why their language model chooses one policy proposal over another.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source ↗