How should an AI-governance proposal evaluate the possibility that some people resist automated decisions because human biases benefit them, while distinguishing that attributed hypothesis from evidence-based concerns about fairness, participation, and legitimacy?

Contribution to the handbook

Priority · Assessed 2026-10-29

This makes a material contribution to evaluating public decision-making: it guards against treating an attributed motive as fact or dismissing concerns about fairness and legitimacy, while prompting comparison of how human discretion and automation may affect groups. The packet reports Zarsky’s explanation for resistance but says Danaher does not endorse it; it also describes possible fairness benefits, possible bias in automated design, and concerns about participation and understanding. These are qualified arguments, not proof of actual effects or opponents’ motives. (s-danaher-threat-of-algocracy-97fa1f966a063bc4; s-danaher-threat-of-algocracy-cf0b97415bdb4af1; s-danaher-threat-of-algocracy-23b1917dc096151f; s-danaher-threat-of-algocracy-f9afba33dcd72217)

Does this challenge the goal?

In-scope gap

The question does not contest the citizen-serving goal or assume that automated decisions are fairer. It asks how to assess objections within the existing scope of fairness, participation, accountability, and evaluating success. It identifies a useful assessment detail: distinguish an attributed explanation of opponents’ motives from evidence about effects and independently grounded procedural concerns.

The question identifies an inquiry, not evidence that opponents have particular motives or that human or automated decisions produce specific group effects.

Authority, accountability and resilience →

Tests, stop conditions and unresolved work →

Identify the proposed explanation for resistance as an attributed hypothesis, not an established account; distinguish evidence about people’s stated reasons from inferences about their motives.

What we can say so far

The possibility that some people resist automated decisions because human biases benefit them should be presented as an attributed hypothesis, not an established account. The supplied source offers a different, attributed forecast: organizations that benefit from AI systems may resist interpretability requirements. It does not establish the proposed explanation about human biases, nor distinguish opponents’ stated reasons from inferences about their motives.

What remains open

• No supplied evidence reports people’s stated reasons for resisting automated decisions.
• No supplied evidence establishes that resistance is motivated by benefiting from human biases.

Assess how human discretion and automated decisions distribute benefits, burdens, errors, and bias across affected groups; distinguish evidence of actual effects from claims about who might benefit.

What we can say so far

The supplied material supports only a qualified account of possible mechanisms, not an assessment of actual distributional effects. Danaher reports Zarsky’s hypothesis that reducing human discretion might protect less powerful groups from human bias, but explicitly does not endorse it as an established result. Another passage describes avoiding some implicit human biases as a possible fairness benefit of automated profiling, while noting that automated systems can also acquire bias during design. Hidalgo and colleagues caution that bias findings from one task should not be assumed to apply to another. These claims do not establish who actually receives benefits or bears burdens, errors, or bias under human versus automated decisions, nor do they establish that any group opposing automation benefits from human discretion.

What remains open

• Empirical comparisons of human and automated decisions, including errors, benefits, burdens, and bias across affected groups.
• Evidence establishing whether, when, and for whom reduced human discretion changes decision outcomes or their distribution.
• Evidence about who benefits from human discretion; the supplied claims about possible beneficiaries or opponents’ motives do not establish actual effects or motives.

Assess concerns such as participation, understanding, and procedural legitimacy on their own grounds, without dismissing them as self-interest or assuming they outweigh other considerations.

What we can say so far

Evaluate participation, public understanding, and procedural legitimacy as independent considerations, rather than treating them as evidence of opponents’ self-interest. The supplied sources frame these concerns as part of the moral and political standing of public decision processes, and explicitly describe weighing possible decision benefits and procedural fairness against losses in understanding and participation. Thus, assess each concern on its own grounds and make any trade-off explicit; do not assume procedural concerns automatically outweigh other considerations, or that they can be dismissed because of an attributed motive. The sources do not provide a rule for assigning weights or resolving particular trade-offs.

What remains open

• No supplied evidence establishes a practical rule for resolving conflicts among participation, understanding, legitimacy, fairness, and decision outcomes.
• The supplied material does not establish the scale or frequency of the deliberation-related time and inclusion challenges it describes.