EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support

Adrian Lindenmeyer iD, Malte Blattmann iD, Stefan Franke, Thomas Neumuth iD, Daniel Schneider iD

Journal of Personalized Medicine · 31.01.2025 · 15 (2) · 58

DOI: 10.3390/jpm15020058 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Personalized Medicine, online erschienen am 31.01.2025. Verfasst von Adrian Lindenmeyer und 4 weiteren Autor:innen. Beteiligte Einrichtung: Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig, Leipzig University, Humboldtstraße 25, 04105 Leipzig, Germ…. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deutsche Forschungsgemeinschaft, Federal Ministry of Education and Research, Federal Ministry of Education and Research of Germany.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Widespread adoption of AI for medical decision-making is still hindered due to ethical and safety-related concerns. For AI-based decision support systems in healthcare settings, it is paramount to be reliable and trustworthy. Common deep learning approaches, however, have the tendency towards overconfidence when faced with unfamiliar or changing conditions. Inappropriate extrapolation beyond well-supported scenarios may have dire consequences highlighting the importance of the reliable estimation of local knowledge uncertainty and its communication to the end user. Materials and Methods: While neural network ensembles (ENNs) have been heralded as a potential solution to these issues for many years, deep learning methods, specifically modeling the amount of knowledge, promise more principled and reliable behavior. This study compares their reliability in clinical applications. We centered our analysis on experiments with low-dimensional toy datasets and the exemplary case study of mortality prediction for intensive care unit hospitalizations using Electronic Health Records (EHRs) from the MIMIC3 study. For predictions on the EHR time series, Encoder-Only Transformer models were employed. Knowledge uncertainty estimation is achieved with both ensemble and Spectral Normalized Neural Gaussian Process (SNGP) variants of the common Transformer model. We designed two datasets to test their reliability in detecting token level and more subtle discrepancies both for toy datasets and an EHR dataset.

Results

While both SNGP and ENN model variants achieve similar prediction performance (AUROC: ≈0.85, AUPRC: ≈0.52 for in-hospital mortality prediction from a selected MIMIC3 benchmark), the former demonstrates improved capabilities to quantify knowledge uncertainty for individual samples/patients. Discussion/Conclusions: Methods including a knowledge model, such as SNGP, offer superior uncertainty estimation compared to traditional stochastic deep learning, leading to more trustworthy and safe clinical decision support.

Schlagwörter

Stichwörter der Autor:innen

clinical decision supportelectronic health recordsepistemic uncertaintyknowledge uncertaintytrustworthy AIuncertainty estimation

Autor:innen und Institutionen

  • Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig, Leipzig University, Humboldtstraße 25, 04105 Leipzig, Germany.
  • Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, Semmelweisstrasse 14, 04103 Leipzig, Germany.

ORCID-Profile: Adrian Lindenmeyer, Malte Blattmann, Thomas Neumuth, Daniel Schneider

Förderung

  • Deutsche Forschungsgemeinschaft
  • Federal Ministry of Education and Research
  • Federal Ministry of Education and Research of Germany

Zitieren

APA

Lindenmeyer, A., Blattmann, M., Franke, S., Neumuth, T., & Schneider, D. (2025). Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support. Journal of Personalized Medicine, 15(2), 58. https://doi.org/10.3390/jpm15020058

Vancouver

Lindenmeyer A, Blattmann M, Franke S, Neumuth T, Schneider D. Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support. Journal of Personalized Medicine. 2025;15(2):58. doi:10.3390/jpm15020058

BibTeX
@article{Lindenmeyer2025,
  author = {Lindenmeyer, Adrian and Blattmann, Malte and Franke, Stefan and Neumuth, Thomas and Schneider, Daniel},
  title = {Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support},
  journal = {Journal of Personalized Medicine},
  year = {2025},
  volume = {15},
  number = {2},
  pages = {58},
  doi = {10.3390/jpm15020058}
}
Kostenlos starten →