EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Clinical Performance and Communication Skills of ChatGPT Versus Physicians in Emergency Medicine: Simulated Patient Study

ChulHyoung Park iD, Min Ho An iD, Gyubeom Hwang iD, Rae Woong Park iD, Juho An iD

JMIR Medical Informatics · 17.07.2025 · 13 · e68409-e68409

DOI: 10.2196/68409 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Medical Informatics, online erschienen am 17.07.2025. Verfasst von ChulHyoung Park und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, Republic of Korea. Hauptthemen laut MEDLINE: Emergency Medicine, Clinical Competence, Patient Simulation, Physicians. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Emergency medicine can benefit from artificial intelligence (AI) due to its unique challenges, such as high patient volume and the need for urgent interventions. However, it remains difficult to assess the applicability of AI systems to real-world emergency medicine practice, which requires not only medical knowledge but also adaptable problem-solving and effective communication skills.

Objective

We aimed to evaluate ChatGPT’s (OpenAI) performance in comparison to human doctors in simulated emergency medicine settings, using the framework of clinical performance examination and written examinations.

Methods

In total, 12 human doctors were recruited to represent the medical professionals. Both ChatGPT and the human doctors were instructed to manage each case like real clinical settings with 12 simulated patients. After the clinical performance examination sessions, the conversation records were evaluated by an emergency medicine professor on history taking, clinical accuracy, and empathy on a 5-point Likert scale. Simulated patients completed a 5-point scale survey including overall comprehensibility, credibility, and concern reduction for each case. In addition, they evaluated whether the doctor they interacted with was similar to a human doctor. An additional evaluation was performed using vignette-based written examinations to assess diagnosis, investigation, and treatment planning. The mean scores from ChatGPT were then compared with those of the human doctors.

Results

ChatGPT scored significantly higher than the physicians in both history-taking (mean score 3.91, SD 0.67 vs mean score 2.67, SD 0.78, P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Emergency MedicineClinical CompetencePatient SimulationPhysiciansCommunicationArtificial IntelligenceHumansMaleFemaleAdultPhysician-Patient RelationsGenerative Artificial Intelligence

Stichwörter der Autor:innen

ChatGPTartificial intelligenceclinical performance examinationclinical reasoningemergency medicineempathyhistory takinglarge language modelpatient experience

Autor:innen und Institutionen

  • Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, Republic of Korea.
  • Center for Biomedical Informatics Research, Ajou University Medical Cencer, Suown, Republic of Korea.
  • Department of Medical Sciences, Graduate School of Ajou University, Suwon, Republic of Korea.
  • BK21 R&E Initiative for Advanced Precision Medicine, Suwon, Republic of Korea.
  • Department of Emergency Medicine, Ajou University School of Medicine, 164 Worldcup-ro, Yeongtong-gu, Suwon, 16499, Republic of Korea, 82 0312195016.

ORCID-Profile: ChulHyoung Park, Min Ho An, Gyubeom Hwang, Rae Woong Park, Juho An

Zitieren

APA

Park, C., An, M. H., Hwang, G., Park, R. W., & An, J. (2025). Clinical Performance and Communication Skills of ChatGPT Versus Physicians in Emergency Medicine: Simulated Patient Study. JMIR Medical Informatics, 13, e68409-e68409. https://doi.org/10.2196/68409

Vancouver

Park C, An MH, Hwang G, Park RW, An J. Clinical Performance and Communication Skills of ChatGPT Versus Physicians in Emergency Medicine: Simulated Patient Study. JMIR Medical Informatics. 2025;13:e68409-e68409. doi:10.2196/68409

BibTeX
@article{Park2025,
  author = {Park, ChulHyoung and An, Min Ho and Hwang, Gyubeom and Park, Rae Woong and An, Juho},
  title = {Clinical Performance and Communication Skills of ChatGPT Versus Physicians in Emergency Medicine: Simulated Patient Study},
  journal = {JMIR Medical Informatics},
  year = {2025},
  volume = {13},
  pages = {e68409--e68409},
  doi = {10.2196/68409}
}
Kostenlos starten →