A starting point for discussion
The idea
Requiring artificial intelligence (AI) decision systems to be understandable may reduce some of their practical benefits and face resistance from businesses or governments that benefit from them.
Place in the reasoning
AI assessment · Connected to reasoning
The author identifies possible loss of practical benefits and resistance from interested businesses or governments as obstacles to requiring understandable systems.
Proposals to improve people’s abilities and require human review have limits
Adds practical and political obstacles to the proposed legal limits; it does not establish that such limits cannot be adopted.
Adapting to computer systems or resisting them can both carry risks
Shows a potential cost and source of opposition relevant to balancing accommodation against other responses.
What this means and why it matters
What this means
Making a system understandable could limit how it works or what it can deliver, while organizations that benefit from current systems may oppose such limits. The author also warns that introducing review rules after these systems have become dominant may be very difficult.
Why it matters
A review rule may be harder to use if it is adopted only after a system is already widely relied on.
Conditions & assumptions
Source coverage: Full readable text of the author manuscript from an accessible mirror; not the publisher typeset version. The author says interpretability requirements could reduce the benefits sought from these systems and meet resistance from interested organizations. Addresses: Whether human review can be required in practice
Questions this connects to
AI-suggested relationships. Authors have not necessarily responded to one another.
How strong is this claim?
AI assessment · not a probability or a final resolution