A starting point for discussion
The idea
The author says it can be hard for people to understand why a computer system that finds patterns in data and uses them—a machine-learning system—produced a particular result. Researchers are working on ways to make these systems easier to understand, but the paper reports that doing so may come at a cost in other areas.
Place in the reasoning
AI assessment · Connected to reasoning
Reports that machine-learning results can be difficult to explain and that interpretability can involve trade-offs.
People need to understand decisions before they can check them
Supports the concern that formal review may not suffice when results are hard to explain, while preserving the stated trade-off qualification.
What this means and why it matters
What this means
The people who build or use a system may not be able to say exactly why it produced a particular result. The paper notes both efforts to improve explanations and possible costs of doing so.
Why it matters
If people cannot get a useful account of a result, it may be harder for them to examine or question decisions based on it.
Conditions & assumptions
Source coverage: Full readable text of the author manuscript from an accessible mirror; not the publisher typeset version. This is a limitation the paper reports from research on computer systems that learn from data, not a finding of the author's own experiment.
Questions this connects to
AI-suggested relationships. Authors have not necessarily responded to one another.
How strong is this claim?
AI assessment · not a probability or a final resolution
The limitation and the ongoing work are both stated, but the passage does not specify the size or kind of costs.