EUVIMEDObjevte EUVIMED
Open access

ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis

John Michael Hoppe iD, Matthias K Auer iD, Anna Strüven iD, Steffen Massberg iD, Christopher Stremmel iD

Journal of Medical Internet Research · 2024-07-08 · 26 · e56110

DOI: 10.2196/56110 ↗

At a glance

Journal article in Journal of Medical Internet Research, published online on 2024-07-08. Written by John Michael Hoppe and 4 co-authors. Institution: Department of Medicine IV, LMU University Hospital, Munich, Germany. Major MEDLINE topics: Emergency Service, Hospital. Cited 100 times so far (Crossref); 11 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

OpenAI’s ChatGPT is a pioneering artificial intelligence (AI) in the field of natural language processing, and it holds significant potential in medicine for providing treatment advice. Additionally, recent studies have demonstrated promising results using ChatGPT for emergency medicine triage. However, its diagnostic accuracy in the emergency department (ED) has not yet been evaluated.

Objective

This study compares the diagnostic accuracy of ChatGPT with GPT-3.5 and GPT-4 and primary treating resident physicians in an ED setting.

Methods

Among 100 adults admitted to our ED in January 2023 with internal medicine issues, the diagnostic accuracy was assessed by comparing the diagnoses made by ED resident physicians and those made by ChatGPT with GPT-3.5 or GPT-4 against the final hospital discharge diagnosis, using a point system for grading accuracy.

Results

The study enrolled 100 patients with a median age of 72 (IQR 58.5-82.0) years who were admitted to our internal medicine ED primarily for cardiovascular, endocrine, gastrointestinal, or infectious diseases. GPT-4 outperformed both GPT-3.5 (P

Subjects

MeSH terms (MEDLINE)

Emergency Service, HospitalHumansRetrospective StudiesAgedFemaleMiddle AgedMaleAged, 80 and overArtificial IntelligencePhysiciansNatural Language ProcessingTriage

Author keywords

AIChatGPTNLPOpenAIaccuracyartificial intelligencediagnosisdiagnostic accuracyemergency departmentemergency medicine triageinternal medicinenatural language processingphysicianphysicianstriage

Authors and institutions

  • Department of Medicine IV, LMU University Hospital, Munich, Germany.
  • Department of Medicine I, LMU University Hospital, Munich, Germany.
  • Munich Heart Alliance Partner Site, Deutsches Zentrum für Herz-Kreislaufforschung (German Centre for Cardiovascular Research), LMU University Hospital, Munich, Germany.

ORCID profiles: John Michael Hoppe, Matthias K Auer, Anna Strüven, Steffen Massberg, Christopher Stremmel

Cite

APA

Hoppe, J. M., Auer, M. K., Strüven, A., Massberg, S., & Stremmel, C. (2024). ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis. Journal of Medical Internet Research, 26, e56110. https://doi.org/10.2196/56110

Vancouver

Hoppe JM, Auer MK, Strüven A, Massberg S, Stremmel C. ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis. Journal of Medical Internet Research. 2024;26:e56110. doi:10.2196/56110

BibTeX
@article{Hoppe2024,
  author = {Hoppe, John Michael and Auer, Matthias K and Strüven, Anna and Massberg, Steffen and Stremmel, Christopher},
  title = {ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  volume = {26},
  pages = {e56110},
  doi = {10.2196/56110}
}
Start for free →