Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support
Journal of Personalized Medicine · 31.01.2025 · 15 (2) · 58
DOI: 10.3390/jpm15020058 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Personalized Medicine, online erschienen am 31.01.2025. Verfasst von Adrian Lindenmeyer und 4 weiteren Autor:innen. Beteiligte Einrichtung: Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig, Leipzig University, Humboldtstraße 25, 04105 Leipzig, Germ…. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deutsche Forschungsgemeinschaft, Federal Ministry of Education and Research, Federal Ministry of Education and Research of Germany.
Abstract (Originalsprache)
Introduction
Widespread adoption of AI for medical decision-making is still hindered due to ethical and safety-related concerns. For AI-based decision support systems in healthcare settings, it is paramount to be reliable and trustworthy. Common deep learning approaches, however, have the tendency towards overconfidence when faced with unfamiliar or changing conditions. Inappropriate extrapolation beyond well-supported scenarios may have dire consequences highlighting the importance of the reliable estimation of local knowledge uncertainty and its communication to the end user. Materials and Methods: While neural network ensembles (ENNs) have been heralded as a potential solution to these issues for many years, deep learning methods, specifically modeling the amount of knowledge, promise more principled and reliable behavior. This study compares their reliability in clinical applications. We centered our analysis on experiments with low-dimensional toy datasets and the exemplary case study of mortality prediction for intensive care unit hospitalizations using Electronic Health Records (EHRs) from the MIMIC3 study. For predictions on the EHR time series, Encoder-Only Transformer models were employed. Knowledge uncertainty estimation is achieved with both ensemble and Spectral Normalized Neural Gaussian Process (SNGP) variants of the common Transformer model. We designed two datasets to test their reliability in detecting token level and more subtle discrepancies both for toy datasets and an EHR dataset.
Results
While both SNGP and ENN model variants achieve similar prediction performance (AUROC: ≈0.85, AUPRC: ≈0.52 for in-hospital mortality prediction from a selected MIMIC3 benchmark), the former demonstrates improved capabilities to quantify knowledge uncertainty for individual samples/patients. Discussion/Conclusions: Methods including a knowledge model, such as SNGP, offer superior uncertainty estimation compared to traditional stochastic deep learning, leading to more trustworthy and safe clinical decision support.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig, Leipzig University, Humboldtstraße 25, 04105 Leipzig, Germany.
- Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, Semmelweisstrasse 14, 04103 Leipzig, Germany.
ORCID-Profile: Adrian Lindenmeyer, Malte Blattmann, Thomas Neumuth, Daniel Schneider
Förderung
- Deutsche Forschungsgemeinschaft
- Federal Ministry of Education and Research
- Federal Ministry of Education and Research of Germany
Zitieren
APA
Lindenmeyer, A., Blattmann, M., Franke, S., Neumuth, T., & Schneider, D. (2025). Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support. Journal of Personalized Medicine, 15(2), 58. https://doi.org/10.3390/jpm15020058
Vancouver
Lindenmeyer A, Blattmann M, Franke S, Neumuth T, Schneider D. Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support. Journal of Personalized Medicine. 2025;15(2):58. doi:10.3390/jpm15020058
BibTeX
@article{Lindenmeyer2025,
author = {Lindenmeyer, Adrian and Blattmann, Malte and Franke, Stefan and Neumuth, Thomas and Schneider, Daniel},
title = {Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support},
journal = {Journal of Personalized Medicine},
year = {2025},
volume = {15},
number = {2},
pages = {58},
doi = {10.3390/jpm15020058}
}
EUVIMED