A starting point for discussion
The idea
The author warns that reliance on computer rules could grow from systems that are fairly easy to follow into systems too complex for people to understand. In public decision-making, these systems could organize and manipulate vast streams of data while being joined to an increasingly complex set of other systems that use computer rules. This could make it harder for people to take part in or understand decisions.
Place in the reasoning
AI assessment · Connected to reasoning
This presents a possible extension from understandable systems to complex, interconnected public systems that may restrict participation and understanding; it is a warning, not an established finding.
Computer systems can shape what people do without taking over
It describes a possible mechanism by which algorithmic organization could limit public participation and understanding, retaining the source’s uncertainty.
People need to understand decisions before they can check them
The described complexity helps explain why public review and understanding could be difficult, without establishing inevitable failure.
What this means and why it matters
What this means
The author describes a possible progression: people may first rely on systems they can understand fairly easily, then on systems that become too complex to follow. He says public systems may also combine large streams of data with other systems that use computer rules.
Why it matters
The concern is that people could have less opportunity to take part in and understand decisions made by public authorities. The passage presents this as a risk, not as a demonstrated outcome.
Conditions & assumptions
Source coverage: Full readable text of the author’s manuscript from an accessible mirror; this is not the final version formatted by the publisher. This is the author's warning about a possible extension of the warehouse example, not evidence that public systems have already reached this state.
Questions this connects to
AI-suggested relationships. Authors have not necessarily responded to one another.