EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access

Early clinical evaluation of a vendor developed pediatric artificial intelligence sepsis model in the emergency department

Swaminathan Kandaswamy iD, Evan W Orenstein iD, Naveen Muthu, Andrea McCarter, Nikolay Braykov, Jonathan M Beus, Edwin Ray, Tal Senior, Sara P Brown, Rebekah Carter, MaryBeth Gleeson, Hannah Thummel und 7 weitere

Journal of the American Medical Informatics Association · 2025

DOI: 10.1093/jamia/ocaf105 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift Journal of the American Medical Informatics Association, 2025. Verfasst von Swaminathan Kandaswamy und 18 weiteren Autor:innen. Beteiligte Einrichtung: Pediatrics, Emory University School of Medicine, Atlanta, GA 30307, United States. Hauptthemen laut MEDLINE: Sepsis, Emergency Service, Hospital, Artificial Intelligence. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 15 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIH HHS, AHRQ HHS, NLM NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To conduct an independent external validation of an implemented vendor-developed emergency department (ED) pediatric sepsis predictive model. Materials and Methods We performed a retrospective cross-sectional study within 2 ED sites of a large pediatric health system between January 1, 2021 and April 1, 2024. A nurse-facing interruptive alert appeared when the model score exceeded the threshold, triggering clinicians to call a sepsis huddle. We compared model predictive performance with vendor-reported performance using definitions that accounted for model threshold and alert timing in clinical practice. Care processes and patient outcome measures included time to first antibiotics, time to first fluid bolus, 30-day mortality, ED to ICU admission rate, and ICU free days.

Results

The pre-intervention cohort consisted of 268 102 ED visits with 741 (0.28%) sepsis cases. The post-intervention cohort consisted of 331 061 ED visits with 1114 (0.34%) sepsis cases. Model predictive performance dropped from vendor-reported performance. Mean time to first antibiotic decreased from 112 to 102 minutes (P = .05, 95% confidence interval of difference, −19.1 to 0.1) and time to first bolus decreased by 16.7 minutes (P = .03, 95% confidence interval difference, −31.8 to −1.5) after the intervention. Decreases in 30-day mortality (6% [45/741] to 4% [52/1114]); ED to ICU admissions (87% [646/741] to 84% [941/1114]), and ICU free days (6 to 5) after the intervention did not meet statistical significance.

Discussion

Implementing the model led to significant reductions in time to fluid bolus and borderline decreases in time to antibiotics, with non-significant changes in mortality and ICU metrics. When implementing an externally developed model, local workflows, documentation patterns, and patient populations make it challenging to generalize published or reported model performance metrics to real world performance.

Conclusion

When tailoring a vendor-developed pediatric ED sepsis model for real-world usage, predictive performance differed substantially. Post-implementation we found improvements in care process measures, suggesting such models may benefit sepsis care when adapted for specific clinical workflows.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SepsisEmergency Service, HospitalArtificial IntelligenceHumansRetrospective StudiesCross-Sectional StudiesChildAnti-Bacterial AgentsChild, PreschoolInfantFemaleMaleTime-to-Treatment

Stichwörter der Autor:innen

clinical evaluationpediatricspredictive modelssepsis

Substanzen

Anti-Bacterial Agents

Autor:innen und Institutionen

  • Pediatrics, Emory University School of Medicine, Atlanta, GA 30307, United States.
  • Information Systems & Technology, Children's Healthcare of Atlanta, Atlanta, GA 30329, United States.
  • Emergency Medicine, Children's Healthcare of Atlanta, Atlanta, GA 30329, United States.
  • Clinical Process Improvement, Children's Healthcare of Atlanta, Atlanta, GA 30329, United States.
  • Critical Care Medicine, Children's Healthcare of Atlanta, Atlanta, GA 30329, United States.

ORCID-Profile: Swaminathan Kandaswamy, Evan W Orenstein, Mark V Mai

Förderung

  • NIH HHS
  • AHRQ HHS
  • NLM NIH HHS
  • NCATS NIH HHS
  • Agency for Healthcare Research and Quality
  • National Center for Advancing Translational Sciences
  • National Institutes of Health

Zitieren

APA

Kandaswamy, S., Orenstein, E. W., Muthu, N., McCarter, A., Braykov, N., Beus, J. M., Ray, E., Senior, T., Brown, S. P., Carter, R., Gleeson, M., Thummel, H., Cheng, J., Bui, T., Blanco, R., Hebbar, K., Fortenberry, J., Iyer, S. B., & Mai, M. V. (2025). Early clinical evaluation of a vendor developed pediatric artificial intelligence sepsis model in the emergency department. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocaf105

Vancouver

Kandaswamy S, Orenstein EW, Muthu N, McCarter A, Braykov N, Beus JM, et al. Early clinical evaluation of a vendor developed pediatric artificial intelligence sepsis model in the emergency department. Journal of the American Medical Informatics Association. 2025. doi:10.1093/jamia/ocaf105

BibTeX
@article{Kandaswamy2025,
  author = {Kandaswamy, Swaminathan and Orenstein, Evan W and Muthu, Naveen and McCarter, Andrea and Braykov, Nikolay and Beus, Jonathan M and Ray, Edwin and Senior, Tal and Brown, Sara P and Carter, Rebekah and Gleeson, MaryBeth and Thummel, Hannah and Cheng, John and Bui, Thuy and Blanco, Reena and Hebbar, Kiran and Fortenberry, James and Iyer, Srikant B and Mai, Mark V},
  title = {Early clinical evaluation of a vendor developed pediatric artificial intelligence sepsis model in the emergency department},
  journal = {Journal of the American Medical Informatics Association},
  year = {2025},
  doi = {10.1093/jamia/ocaf105}
}
Kostenlos starten →