EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance

Robert R. Hoffman, Shane T. Mueller, Gary Klein, Jordan Litman

Frontiers in Computer Science · 06.02.2023 · 5

DOI: 10.3389/fcomp.2023.1096257 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Computer Science, online erschienen am 06.02.2023. Verfasst von Robert R. Hoffman und 3 weiteren Autor:innen. Die Arbeit wurde bisher 287-mal zitiert (Crossref) und nennt 164 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Defense Advanced Research Projects Agency.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

If a user is presented an AI system that portends to explain how it works, how do we know whether the explanation works and the user has achieved a pragmatic understanding of the AI? This question entails some key concepts of measurement such as explanation goodness and trust. We present methods for enabling developers and researchers to: (1) Assess the a priori goodness of explanations, (2) Assess users' satisfaction with explanations, (3) Reveal user's mental model of an AI system, (4) Assess user's curiosity or need for explanations, (5) Assess whether the user's trust and reliance on the AI are appropriate, and finally, (6) Assess how the human-XAI work system performs. The methods we present derive from our integration of extensive research literatures and our own psychometric evaluations. We point to the previous research that led to the measurement scales which we aggregated and tailored specifically for the XAI context. Scales are presented in sufficient detail to enable their use by XAI researchers. For Mental Model assessment and Work System Performance, XAI researchers have choices. We point to a number of methods, expressed in terms of methods' strengths and weaknesses, and pertinent measurement issues.

Förderung

  • Defense Advanced Research Projects Agency

Zitieren

APA

Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2023). Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance. Frontiers in Computer Science, 5. https://doi.org/10.3389/fcomp.2023.1096257

Vancouver

Hoffman RR, Mueller ST, Klein G, Litman J. Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance. Frontiers in Computer Science. 2023;5. doi:10.3389/fcomp.2023.1096257

BibTeX
@article{Hoffman2023,
  author = {Hoffman, Robert R. and Mueller, Shane T. and Klein, Gary and Litman, Jordan},
  title = {Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance},
  journal = {Frontiers in Computer Science},
  year = {2023},
  volume = {5},
  doi = {10.3389/fcomp.2023.1096257}
}
Kostenlos starten →