Transportability of bacterial infection prediction models for critically ill patients
Journal of the American Medical Informatics Association · 22.12.2023 · 31 (1) · 98-108
DOI: 10.1093/jamia/ocad174 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 30.08.2023. Verfasst von Garrett Eickelberg und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Preventive Medicine (Health & Biomedical Informatics), Feinberg School of Medicine, Chicago, IL 60611, United States. Hauptthemen laut MEDLINE: Critical Illness, Bacterial Infections. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIH HHS, NLM NIH HHS, NICHD NIH HHS.
Abstract (Originalsprache)
Objective
Bacterial infections (BIs) are common, costly, and potentially life-threatening in critically ill patients. Patients with suspected BIs may require empiric multidrug antibiotic regimens and therefore potentially be exposed to prolonged and unnecessary antibiotics. We previously developed a BI risk model to augment practices and help shorten the duration of unnecessary antibiotics to improve patient outcomes. Here, we have performed a transportability assessment of this BI risk model in 2 tertiary intensive care unit (ICU) settings and a community ICU setting. We additionally explored how simple multisite learning techniques impacted model transportability.
Methods
Patients suspected of having a community-acquired BI were identified in 3 datasets: Medical Information Mart for Intensive Care III (MIMIC), Northwestern Medicine Tertiary (NM-T) ICUs, and NM “community-based” ICUs. ICU encounters from MIMIC and NM-T datasets were split into 70/30 train and test sets. Models developed on training data were evaluated against the NM-T and MIMIC test sets, as well as NM community validation data.
Results
During internal validations, models achieved AUROCs of 0.78 (MIMIC) and 0.81 (NM-T) and were well calibrated. In the external community ICU validation, the NM-T model had robust transportability (AUROC 0.81) while the MIMIC model transported less favorably (AUROC 0.74), likely due to case-mix differences. Multisite learning provided no significant discrimination benefit in internal validation studies but offered more stability during transport across all evaluation datasets.
Discussion
These results suggest that our BI risk models maintain predictive utility when transported to external cohorts.
Conclusion
Our findings highlight the importance of performing external model validation on myriad clinically relevant populations prior to implementation.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Substanzen
Autor:innen und Institutionen
- Department of Preventive Medicine (Health & Biomedical Informatics), Feinberg School of Medicine, Chicago, IL 60611, United States.
- Departments of Pediatrics (Critical Care), Chicago, IL 60611, United States.
ORCID-Profile: Garrett Eickelberg, Lazaro Nelson Sanchez-Pinto, Yuan Luo
Förderung
- NIH HHS
- NLM NIH HHS
- NICHD NIH HHS
- NCATS NIH HHS
- National Institutes of Health
- National Library of Medicine
- National Institute of Child Health & Human Development
Zitieren
APA
Eickelberg, G., Sanchez-Pinto, L. N., Kline, A. S., & Luo, Y. (2023). Transportability of bacterial infection prediction models for critically ill patients. Journal of the American Medical Informatics Association, 31(1), 98-108. https://doi.org/10.1093/jamia/ocad174
Vancouver
Eickelberg G, Sanchez-Pinto LN, Kline AS, Luo Y. Transportability of bacterial infection prediction models for critically ill patients. Journal of the American Medical Informatics Association. 2023;31(1):98-108. doi:10.1093/jamia/ocad174
BibTeX
@article{Eickelberg2023,
author = {Eickelberg, Garrett and Sanchez-Pinto, Lazaro Nelson and Kline, Adrienne Sarah and Luo, Yuan},
title = {Transportability of bacterial infection prediction models for critically ill patients},
journal = {Journal of the American Medical Informatics Association},
year = {2023},
volume = {31},
number = {1},
pages = {98--108},
doi = {10.1093/jamia/ocad174}
}
EUVIMED