How Explainable Artificial Intelligence Can Increase or Decrease Clinicians’ Trust in AI Applications in Health Care: Systematic Review
Wie Explainable Artificial Intelligence das Vertrauen von Kliniker:innen in KI-Anwendungen im Gesundheitswesen erhöhen oder senken kann: systematische Übersichtsarbeit
JMIR AI · 30.10.2024 · 3 · e53207
DOI: 10.2196/53207 ↗Auf einen Blick
Die Übersichtsarbeit zu 10 Studien zeigt: Explainable AI (XAI) kann das Vertrauen von Kliniker:innen in KI erhöhen, aber auch senken. Entscheidend war die Qualität der Erklärungen.
Fragestellung
Welchen Einfluss hat Explainable AI auf das Vertrauen von Kliniker:innen in KI-gestützte klinische Entscheidungen?
Methode
Systematische Übersichtsarbeit nach PRISMA; Suche in PubMed und Web of Science. Eingeschlossen wurden Studien, die den Einfluss von XAI auf Vertrauen empirisch maßen.
Ergebnisse
10 Studien erfüllten die Einschlusskriterien. 5 berichteten mehr Vertrauen mit XAI, 3 fanden keinen signifikanten Effekt, 2 zeigten je nach Komplexität und Kohärenz Zu- oder Abnahme des Vertrauens.
Einschränkungen
Verzerrungsrisiko in allen Studien moderat bis moderat-hoch; Vertrauen vor allem kognitiv operationalisiert, nur 2 Studien nutzten auch affektbasierte Maße.
Schlussfolgerung der Autor:innen
Zu viel Vertrauen in falsche KI-Empfehlungen kann die klinische Genauigkeit beeinträchtigen, ebenso wie Misstrauen gegenüber korrekten Empfehlungen. XAI sollte so gestaltet werden, dass ein angemessenes Vertrauensniveau entsteht.
Kernaussagen
- XAI erhöhte in 5 Studien das Vertrauen von Kliniker:innen
- 3 Studien zeigten keinen signifikanten Effekt auf Vertrauen
- Erklärungsqualität kann Vertrauen erhöhen oder mindern
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Übersichtsarbeit in der Zeitschrift JMIR AI, online erschienen am 30.10.2024. Verfasst von Rikard Rosenbacke und 3 weiteren Autor:innen. Beteiligte Einrichtung: Centre for Corporate Governance, Department of Accounting, Copenhagen Business School, Frederiksberg, Denmark. Die Arbeit wurde bisher 346-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Artificial intelligence (AI) has significant potential in clinical practice. However, its “black box” nature can lead clinicians to question its value. The challenge is to create sufficient trust for clinicians to feel comfortable using AI, but not so much that they defer to it even when it produces results that conflict with their clinical judgment in ways that lead to incorrect decisions. Explainable AI (XAI) aims to address this by providing explanations of how AI algorithms reach their conclusions. However, it remains unclear whether such explanations foster an appropriate degree of trust to ensure the optimal use of AI in clinical practice.
Objective
This study aims to systematically review and synthesize empirical evidence on the impact of XAI on clinicians’ trust in AI-driven clinical decision-making.
Methods
A systematic review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, searching PubMed and Web of Science databases. Studies were included if they empirically measured the impact of XAI on clinicians’ trust using cognition- or affect-based measures. Out of 778 articles screened, 10 met the inclusion criteria. We assessed the risk of bias using standard tools appropriate to the methodology of each paper.
Results
The risk of bias in all papers was moderate or moderate to high. All included studies operationalized trust primarily through cognitive-based definitions, with 2 also incorporating affect-based measures. Out of these, 5 studies reported that XAI increased clinicians’ trust compared with standard AI, particularly when the explanations were clear, concise, and relevant to clinical practice. In addition, 3 studies found no significant effect of XAI on trust, and the presence of explanations does not automatically improve trust. Notably, 2 studies highlighted that XAI could either enhance or diminish trust, depending on the complexity and coherence of the provided explanations. The majority of studies suggest that XAI has the potential to enhance clinicians’ trust in recommendations generated by AI. However, complex or contradictory explanations can undermine this trust. More critically, trust in AI is not inherently beneficial, as AI recommendations are not infallible. These findings underscore the nuanced role of explanation quality and suggest that trust can be modulated through the careful design of XAI systems.
Conclusions
Excessive trust in incorrect advice generated by AI can adversely impact clinical accuracy, just as can happen when correct advice is distrusted. Future research should focus on refining both cognitive and affect-based measures of trust and on developing strategies to achieve an appropriate balance in terms of trust, preventing both blind trust and undue skepticism. Optimizing trust in AI systems is essential for their effective integration into clinical practice.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Centre for Corporate Governance, Department of Accounting, Copenhagen Business School, Frederiksberg, Denmark.
- Department of Medical Sciences, Clinical Microbiology, Uppsala University, Uppsala, Sweden.
- European Observatory on Health Systems and Policies, London School of Hygiene & Tropical Medicine, London, United Kingdom.
- Department of Social and Political Sciences, Bocconi University, Milano, Italy.
ORCID-Profile: Rikard Rosenbacke, Åsa Melhus, Martin McKee, David Stuckler
Zitieren
APA
Rosenbacke, R., Melhus, Å., McKee, M., & Stuckler, D. (2024). How Explainable Artificial Intelligence Can Increase or Decrease Clinicians’ Trust in AI Applications in Health Care: Systematic Review. JMIR AI, 3, e53207. https://doi.org/10.2196/53207
Vancouver
Rosenbacke R, Melhus Å, McKee M, Stuckler D. How Explainable Artificial Intelligence Can Increase or Decrease Clinicians’ Trust in AI Applications in Health Care: Systematic Review. JMIR AI. 2024;3:e53207. doi:10.2196/53207
BibTeX
@article{Rosenbacke2024,
author = {Rosenbacke, Rikard and Melhus, Åsa and McKee, Martin and Stuckler, David},
title = {How Explainable Artificial Intelligence Can Increase or Decrease Clinicians’ Trust in AI Applications in Health Care: Systematic Review},
journal = {JMIR AI},
year = {2024},
volume = {3},
pages = {e53207},
doi = {10.2196/53207}
}
EUVIMED