ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis
Journal of Medical Internet Research · 2024-07-08 · 26 · e56110
DOI: 10.2196/56110 ↗At a glance
Journal article in Journal of Medical Internet Research, published online on 2024-07-08. Written by John Michael Hoppe and 4 co-authors. Institution: Department of Medicine IV, LMU University Hospital, Munich, Germany. Major MEDLINE topics: Emergency Service, Hospital. Cited 100 times so far (Crossref); 11 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
OpenAI’s ChatGPT is a pioneering artificial intelligence (AI) in the field of natural language processing, and it holds significant potential in medicine for providing treatment advice. Additionally, recent studies have demonstrated promising results using ChatGPT for emergency medicine triage. However, its diagnostic accuracy in the emergency department (ED) has not yet been evaluated.
Objective
This study compares the diagnostic accuracy of ChatGPT with GPT-3.5 and GPT-4 and primary treating resident physicians in an ED setting.
Methods
Among 100 adults admitted to our ED in January 2023 with internal medicine issues, the diagnostic accuracy was assessed by comparing the diagnoses made by ED resident physicians and those made by ChatGPT with GPT-3.5 or GPT-4 against the final hospital discharge diagnosis, using a point system for grading accuracy.
Results
The study enrolled 100 patients with a median age of 72 (IQR 58.5-82.0) years who were admitted to our internal medicine ED primarily for cardiovascular, endocrine, gastrointestinal, or infectious diseases. GPT-4 outperformed both GPT-3.5 (P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Medicine IV, LMU University Hospital, Munich, Germany.
- Department of Medicine I, LMU University Hospital, Munich, Germany.
- Munich Heart Alliance Partner Site, Deutsches Zentrum für Herz-Kreislaufforschung (German Centre for Cardiovascular Research), LMU University Hospital, Munich, Germany.
ORCID profiles: John Michael Hoppe, Matthias K Auer, Anna Strüven, Steffen Massberg, Christopher Stremmel
Cite
APA
Hoppe, J. M., Auer, M. K., Strüven, A., Massberg, S., & Stremmel, C. (2024). ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis. Journal of Medical Internet Research, 26, e56110. https://doi.org/10.2196/56110
Vancouver
Hoppe JM, Auer MK, Strüven A, Massberg S, Stremmel C. ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis. Journal of Medical Internet Research. 2024;26:e56110. doi:10.2196/56110
BibTeX
@article{Hoppe2024,
author = {Hoppe, John Michael and Auer, Matthias K and Strüven, Anna and Massberg, Steffen and Stremmel, Christopher},
title = {ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis},
journal = {Journal of Medical Internet Research},
year = {2024},
volume = {26},
pages = {e56110},
doi = {10.2196/56110}
}
EUVIMED