DARPA"XAI"Literature"Review"" " " " " " " " " p."
"
"
Explanation in Human-AI Systems
Executive Summary
This is an integrative review that address the question, "What makes for a good
explanation?" with reference to AI systems. Pertinent literatures are vast. Thus, this review is
necessarily selective. That said, most of the key concepts and issues are exressed in this Report.
The Report encapsulates the history of computer science efforts to create systems that explain
and instruct (intelligent tutoring systems and expert systems). The Report expresses the
explainability issues and challenges in modern AI, and presents capsule views of the leading
psychological theories of explanation. Certain articles stand out by virtue of their particular
relevance to XAI, and their methods, results, and key points are highlighted.
It is recommended that AI/XAI researchers be encouraged to include in their research
reports fuller details on their empirical or experimental methods, in the fashion of experimental
psychology research reports: details on Participants, Instructions, Procedures, Tasks, Dependent
Variables (operational definitions of the measures and metrics), Independent Variables
(conditions), and Control Conditions.
In the papers reviewed in this Report one can find methodological guidance for the
evaluation of XAI systems. But the Report highlights some noteworthy considerations: The
differences between global and local explanations, the need to evaluate the performance of the
human-machine work system (and not just the performance of the AI or the performance of the
users), the need to recognize that experiment procedures tacitly impose on the user the burden of
self-explanation.
Corrective/contrastive user tasks support self-explanation or explanation-as-exploration.
Tasks that involve human-AI interactivity and co-adaptation, such as bug or oddball detection,
hold promise for XAI evaluation since they too conform to the notions of "explanation-as-
exploration" and explanation as a co-adaptive dialog process. Tasks that involve predicting the
AI's determinations, combined with post-experimental interviews, hold promise for the study of
mental models in the XAI context.
评论0