Not All Clinical AI Monitoring Systems Are Created Equal: Review and Recommendations
NEJM AI · 23.01.2025 · 2 (2)
DOI: 10.1056/aira2400657 ↗Kurz gefasst
Übersichtsarbeit in der Zeitschrift NEJM AI, erschienen am 23.01.2025. Verfasst von Jean Feng und 3 weiteren Autor:innen. Beteiligte Einrichtung: University of California, San Francisco. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 77 Literaturangaben.
Abstract (Originalsprache)
AbstractWhile practices for the initial evaluation of clinical artificial intelligence (AI) algorithms are well established, there is little consensus on how to design effective monitoring systems for the post-deployment setting. In real-world case studies, design choices are often driven by practical constraints rather than their impact on the performance of the monitoring system. This narrative review critically examines the key decisions that shape the performance of AI monitoring systems, including the selection of monitoring criteria, the choice of data sources, and the statistical procedures employed. Our findings reveal significant variation in the designed systems, often with little transparency regarding how their design choices affect monitoring performance. To provide a more structured approach to designing monitoring systems that are both effective and practical, we introduce a road map for navigating the many options available. We bootstrap efforts in clinical AI monitoring by highlighting tools from three related fields that face similarly complex challenges: quality management, clinical trials, and real-world evidence generation.
Autor:innen und Institutionen
- University of California, San Francisco.
- University of California, San Diego.
ORCID-Profile: Jean Feng, Fan Xia, Karandeep Singh, Romain Pirracchio
Zitieren
APA
Feng, J., Xia, F., Singh, K., & Pirracchio, R. (2025). Not All Clinical AI Monitoring Systems Are Created Equal: Review and Recommendations. NEJM AI, 2(2). https://doi.org/10.1056/aira2400657
Vancouver
Feng J, Xia F, Singh K, Pirracchio R. Not All Clinical AI Monitoring Systems Are Created Equal: Review and Recommendations. NEJM AI. 2025;2(2). doi:10.1056/aira2400657
BibTeX
@article{Feng2025,
author = {Feng, Jean and Xia, Fan and Singh, Karandeep and Pirracchio, Romain},
title = {Not All Clinical AI Monitoring Systems Are Created Equal: Review and Recommendations},
journal = {NEJM AI},
year = {2025},
volume = {2},
number = {2},
doi = {10.1056/aira2400657}
}
EUVIMED