A starting point for discussion
The idea
Danaher says difficulty understanding how an algorithm shapes a decision may be as serious as, or more serious than, the fact that its role is hidden. He uses opacity to mean that people may find a process hard to understand through reasoning by people.
Place in the reasoning
AI assessment · Connected to reasoning
This distinguishes not knowing that an algorithm is involved from difficulty understanding how it shapes a decision.
People need to understand decisions before they can check them
Difficulty understanding decision processes helps explain why disclosure or formal human review may not enable meaningful scrutiny.
Computer systems can shape what people do without taking over
It describes opacity as a possible constraint on understanding, without asserting that every algorithmic system is opaque.
What this means and why it matters
What this means
People might know that an algorithm is being used and still be unable to understand how it shapes a public decision.
Why it matters
Simply saying that an algorithm is involved may not address the author’s main concern. The passages do not test whether people can understand particular systems.
Conditions & assumptions
Source coverage: Full readable text of the author manuscript from an accessible mirror; not the publisher typeset version. The author distinguishes not knowing that an algorithm is involved from being unable to understand how a decision process works. Addresses: Hidden algorithm use and difficulty understanding.
Questions this connects to
AI-suggested relationships. Authors have not necessarily responded to one another.
How strong is this claim?
AI assessment · not a probability or a final resolution
The author explicitly says that difficulty understanding may be as serious as, or more serious than, hidden use.