Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives
PLOS Digital Health · 14.05.2026 · 5 (5) · e0001417
DOI: 10.1371/journal.pdig.0001417 ↗Kurz gefasst
Scoping Review in der Zeitschrift PLOS Digital Health, online erschienen am 14.05.2026. Verfasst von Bethany A Van Dort und 4 weiteren Autor:innen. Beteiligte Einrichtung: eHealth Living & Learning Lab Amsterdam, Department of Medical Informatics, Amsterdam UMC - University of Amsterdam, Amsterdam, the Netherlands. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Explainable Artificial Intelligence (XAI) has the potential to enhance clinical decision support (CDS) systems however, it remains unclear how XAI systems are perceived by healthcare professionals in hospital settings, and if new challenges arise as a result of explanations. This scoping review aimed to understand healthcare professionals’ perceptions of CDS systems with XAI in the hospital setting; specifically, the drivers of acceptance and use, explainability needs, and design preferences. Databases were searched. Studies were included if they reported qualitative findings on health professionals’ perceptions of XAI-enabled CDS systems used in hospital settings. MEDLINE, Embase, and Web of Science were searched, and reference lists were screened for additional papers. Study characteristics and health professional perceptions were extracted and inductively coded. A quality assessment was performed using the CASP checklist. Sixteen studies were identified. Included studies primarily focused on ML-based CDS systems for predicting various clinical outcomes. Most studies used feature importance or model agnostic techniques like SHapley Additive exPlanations (SHAP). Overall, healthcare professionals perceived CDS systems with XAI as useful for supporting clinical tasks, decision-making, and teamwork. Acceptance was influenced by integration into workflows, performance, data quality, and alignment with clinical knowledge. Concerns were raised about overreliance and reduced professional autonomy. Health professionals predominantly used explanations to validate outputs, and desired actionable information from systems. SHAP plots and visualizations were difficult to interpret. Participants preferred explanation designs that included concise, high-level information, and simple plots for quick interpretation. Clear visual indicators such as colour coding, contextual patient data, and aggregating similar features also aided interpretation. Poorly designed XAI explanations can hinder understanding and increase cognitive burden in busy clinical settings. Future research should optimise the design and delivery of explanations, so clinicians can appropriately trust in XAI CDS systems and feel confident in their clinical decision making.
Autor:innen und Institutionen
- eHealth Living & Learning Lab Amsterdam, Department of Medical Informatics, Amsterdam UMC - University of Amsterdam, Amsterdam, the Netherlands.
- Amsterdam Public Health Research Institute, Amsterdam UMC, Amsterdam, the Netherlands.
- Department of Medical Informatics, Amsterdam UMC - University of Amsterdam, Amsterdam, the Netherlands.
ORCID-Profile: Bethany A Van Dort
Zitieren
APA
Van Dort, B. A., Engelsma, T., Sneekes, P., Peute, L., & Medlock, S. (2026). Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives. PLOS Digital Health, 5(5), e0001417. https://doi.org/10.1371/journal.pdig.0001417
Vancouver
Van Dort BA, Engelsma T, Sneekes P, Peute L, Medlock S. Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives. PLOS Digital Health. 2026;5(5):e0001417. doi:10.1371/journal.pdig.0001417
BibTeX
@article{VanDort2026,
author = {Van Dort, Bethany A and Engelsma, Thomas and Sneekes, Pamela and Peute, Linda and Medlock, Stephanie},
title = {Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives},
journal = {PLOS Digital Health},
year = {2026},
volume = {5},
number = {5},
pages = {e0001417},
doi = {10.1371/journal.pdig.0001417}
}
EUVIMED