Diagnostic Accuracy of a Large Language Model (ChatGPT-4) for Patients Admitted to a Community Hospital Medical Intensive Care Unit: A Retrospective Case Study
Diagnostische Genauigkeit eines Large Language Model (ChatGPT-4) bei Patient:innen einer medizinischen Intensivstation eines Community-Hospitals: retrospektive Fallstudie
Journal of Intensive Care Medicine · 2026-05 · 41 (5) · 413-420
DOI: 10.1177/08850666251368270 ↗Auf einen Blick
Bei 120 zufällig ausgewählten Fällen war die diagnostische Genauigkeit von ChatGPT-4 ähnlich wie die der Ärzt:innen. Die Übereinstimmung war nur moderat.
Fragestellung
Kann ChatGPT-4 bei kritisch kranken Patient:innen auf einer medizinischen Intensivstation eine akkurate Diagnose aus einem realen Falltext stellen?
Methode
Retrospektive Fallstudie mit 120 zufällig ausgewählten Fällen; redigierte Anamnese/Status in ChatGPT-4; Vergleich mit Ärzt:innen-Diagnosen und blinder longitudinaler Chart-Review als Referenz.
Ergebnisse
Diagnostische Genauigkeit: 88,3 % bei Ärzt:innen und 85,0 % bei ChatGPT-4; kein signifikanter Unterschied (McNemar p=0,249). Kappa für die Übereinstimmung: 0,57 (95 %-KI 0,35–0,79).
Schlussfolgerung der Autor:innen
Die Ergebnisse sprechen dafür, dass ChatGPT-4 in diesem Setting eine mit Fachärzt:innen vergleichbare diagnostische Genauigkeit erreichte. Die Übereinstimmung war nur moderat; eine prospektive Studie sei nötig.
Kernaussagen
- Diagnostische Genauigkeit: 88,3 % vs. 85,0 %
- Übereinstimmung moderat: Kappa 0,57 (95 %-KI 0,35–0,79)
- Vergleich mit blinder longitudinaler Chart-Review als Referenz
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Fachartikel in der Zeitschrift Journal of Intensive Care Medicine, online erschienen am 17.08.2025. Verfasst von Jassimran Singh und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Internal Medicine, Saint Vincent Hospital, Worcester, MA 01608, USA. Hauptthemen laut MEDLINE: Intensive Care Units, Artificial Intelligence. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 22 Literaturangaben.
Themen: Künstliche Intelligenz in der Pflege, Intensivpflege
Abstract (Originalsprache)
Background
The future of artificial intelligence in medicine includes the use of machine learning and large language models to improve diagnostic accuracy, as a point-of-care tool, at the time of admission to an acute care hospital. The large language model, ChatGPT-4, has been shown to diagnose complex medical conditions with accuracies comparable to experienced clinicians, however, most published studies involved curated cases or examination-like questions and are not point-of-care. To test the hypothesis that ChatGPT-4 can make an accurate medical diagnosis using real-world medical cases and a convenient cut and paste strategy, we performed a retrospective case study involving critically ill patients admitted to a community hospital medical intensive care unit.
Methods
A redacted H&P was essentially cut and pasted into ChatGPT-4 with uniform instructions to make a leading diagnosis and a list of 5 possibilities as a differential diagnosis. All features that could be used to identify patients were removed to ensure privacy and HIPAA compliance. The ChatGPT-4 diagnoses were compared with critical care physician diagnoses using a blinded longitudinal chart review as the ground truth diagnosis.
Results
A total of 120 randomly selected cases were included in the study. The diagnostic accuracy was 88.3% for physicians and 85.0% for ChatGPT-4, with no significant difference by McNemar testing (p-value of 0.249). The agreement between physician diagnosis and ChatGPT-4 diagnosis was moderate, 0.57 (95% CI: 0.35–0.79), based on Cohen's kappa statistic.
Conclusion
These results suggest that ChatGTP-4 achieved diagnostic accuracy comparable to board certified physicians in the context of critically ill patients admitted to a community medical intensive care unit. Furthermore, the agreement was only moderate, suggesting that there may be complementary ways of combining the diagnostic acumen of physicians and ChatGPT-4 to improve overall accuracy. A prospective study would be necessary to determine if ChatGPT-4 could improve patient outcomes as a point-of-care tool at the time of admission.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Internal Medicine, Saint Vincent Hospital, Worcester, MA 01608, USA.
- Department of Pulmonary and Critical Care, UMass Memorial, Worcester, MA 01608, USA.
- Department of Nursing, Saint Vincent Hospital, Worcester, MA 01608, USA.
- Division of Pulmonary and Critical Care, Department of Medicine, Saint Vincent Hospital, Worcester, MA 01608, USA.
ORCID-Profile: Jassimran Singh, Charmi Bhanushali
Zitieren
APA
Singh, J., Bohra, R., Mukhtiar, V., Fernandes, W., Bhanushali, C., Chinnamuthu, R., Kanamgode, S. S., Ellis, J., & Silverman, E. (2026). Diagnostic Accuracy of a Large Language Model (ChatGPT-4) for Patients Admitted to a Community Hospital Medical Intensive Care Unit: A Retrospective Case Study. Journal of Intensive Care Medicine, 41(5), 413-420. https://doi.org/10.1177/08850666251368270
Vancouver
Singh J, Bohra R, Mukhtiar V, Fernandes W, Bhanushali C, Chinnamuthu R, et al. Diagnostic Accuracy of a Large Language Model (ChatGPT-4) for Patients Admitted to a Community Hospital Medical Intensive Care Unit: A Retrospective Case Study. Journal of Intensive Care Medicine. 2026;41(5):413-420. doi:10.1177/08850666251368270
BibTeX
@article{Singh2026,
author = {Singh, Jassimran and Bohra, Rhea and Mukhtiar, Vaibhavi and Fernandes, Warren and Bhanushali, Charmi and Chinnamuthu, Rajaeaswaran and Kanamgode, Shihla Shireen and Ellis, June and Silverman, Eric},
title = {Diagnostic Accuracy of a Large Language Model (ChatGPT-4) for Patients Admitted to a Community Hospital Medical Intensive Care Unit: A Retrospective Case Study},
journal = {Journal of Intensive Care Medicine},
year = {2026},
volume = {41},
number = {5},
pages = {413--420},
doi = {10.1177/08850666251368270}
}
EUVIMED