What access and verification arrangements would let a personal AI helper explain an institution’s algorithmic decision to its user without replacing dependence on the institution with dependence on the helper’s maker?

Contribution to the handbook

Priority · Assessed 2026-10-28

It identifies a concrete explanation and recourse mechanism, a risk of shifting power to tool-makers, and a test of whether users can actually understand and contest decisions. The evidence also distinguishes explanation from recording alone and flags unequal data access.

Does this challenge the goal?

In-scope gap

It examines whether a personal AI helper could aid people in checking institutional decisions without creating a new dependence on the helper’s maker. This fits citizen scrutiny and accountability within the definition; it does not assume that helpers are required or effective.

The evidence describes the helper as a proposal and identifies potential dependence and data-access problems, but does not establish their frequency or show that the proposed checks work.

Applying laws and acting →

Judging situations and resolving disputes →

Authority, accountability and resilience →

Tests, stop conditions and unresolved work →

Identify the decision-specific data, evidence, factors, and reasons needed for an explanation; specify what the institution, user, or another authorized source could supply and what access may be withheld.

What we can say so far

For a decision affecting the user, the helper would need a case-specific account: notice of the decision, the evidence and user data relied on, the factors applied, and the reasons connecting those factors to the outcome. The institution would be the source for its decision record, the factors it used, and its stated reasoning; the user could supply their own records to check factual inputs; and an authorized information-access process might supply institutional rules. The helper could compare these materials and flag discrepancies, but a recording or a general description of the algorithm’s purpose would not establish why this decision was made. Trade-secret and security restrictions may withhold institutional material, and some methods may resist intelligible review even with access. In those cases the helper should identify what it cannot verify rather than present an inferred explanation as the institution’s reason.

What remains open

• What minimum decision-specific record would make the explanation adequate for different systems.
• What lawful access an affected user or authorized reviewer would have to protected data, rules, or decision records.
• How to verify a stated explanation when the decision method is uninterpretable or depends on connected systems.

Compare whether a helper-mediated explanation actually improves users’ ability to identify errors and understand and contest decision factors against an institutional explanation or independent review, rather than inferring understanding from an intelligible-sounding output.

What we can say so far

The supplied material does not establish that a helper-mediated explanation improves users’ ability to identify errors, understand decision factors, or contest a decision compared with an institutional explanation or independent review. Danaher’s argument gives reasons not to infer understanding from a useful or intelligible-sounding helper output: better human–AI performance need not mean the person understands the process, and a record of a decision may not reveal its reasons. A comparison would need to test what affected users can actually identify, explain, and contest under each arrangement, rather than rate the apparent clarity of the output.

What remains open

• No supplied study measures these user outcomes across the three explanation or review arrangements.
• The supplied material does not establish which arrangement works better when institutional data access and control of the helper are unequal.

Assess how unequal access to institutional datasets and maker control over the helper’s design, updates, and outputs could limit or distort the account, and which proposed access and verification arrangements would expose those limits without presuming that provider choice alone solves them.

What we can say so far

A personal helper could help a user examine an algorithmic decision affecting them, but its account would be limited if the institution controls the relevant data or if the helper’s maker controls what the tool can inspect and say. A workable arrangement would give the affected user, or a helper acting on their behalf, access to the decision-specific inputs, applicable rules and records needed to check the explanation, subject to justified privacy and security limits. The user would also need ways to check the helper’s account against institutional records and to challenge discrepancies. Transparency about the helper’s code, training data, governance and changes to its outputs would help expose maker influence; independent review could provide another check. These are proposed design requirements, not demonstrated safeguards. Danaher proposes pairing people’s enhanced ability to understand decisions with a legal requirement that systems be understandable to them. Hidalgo and coauthors suggest provider choice may help, but warn that ownership and funding arrangements remain open to capture. Switching helpers therefore cannot substitute for access to the institution’s evidence or scrutiny of the helper itself.

What remains open

• No supplied source specifies which decision records institutions must provide, who can inspect protected material, or how privacy, trade-secret and security limits should be adjudicated.
• No supplied source establishes how to verify a helper’s explanation against the institution’s decision, detect consequential changes in the helper’s design or outputs, or enforce a right to challenge either account.
• The feasibility and effectiveness of the proposed access, transparency and human-understandability requirements remain untested in the supplied evidence.