Performance of GPT-4o and Claude in Medical Ethics Scenarios: Comparative Study
JMIR Medical Education · 23.07.2026 · 12 · e70199-e70199
DOI: 10.2196/70199 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift JMIR Medical Education, online erschienen am 23.07.2026. Verfasst von Karishma R Desai und 2 weiteren Autor:innen. Beteiligte Einrichtung: Division of Hand Surgery, Department of Orthopaedic Surgery, Emory University, 80 Jesse Hill Jr Drive SE, Atlanta, GA, 30303, United States, 1 404-77…. Hauptthemen laut MEDLINE: Ethics, Medical. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Themen: Ethik in der Pflege
Abstract (Originalsprache)
Background
The emergence of AI technology has sparked curiosity regarding the capabilities of large language models (LLMs) in the field of medicine. Minimal research exists regarding the proficiency of various AI models in ethics scenarios, specifically in specialty-based scenarios.
Objective
This study aimed to compare the performance of GPT-4o and Claude Sonnet 4 on ethics questions with that of medical students and orthopedic residents.
Methods
A total of 200 ethical or legal scenario questions were randomly selected from question banks targeted for third- and fourth-year medical students (UWorld, AMBOSS) and orthopedic residents (OrthoBullets). Questions at the medical student level were exclusively text-based, while resident-level questions included text-based questions accompanied by images. Each question was entered identically into each AI model 3 separate times. If answers varied between trials, the answer provided most frequently by the model was used as the selected answer.
Results
GPT-4o correctly answered 140 (70%) of 200 questions, which was similar to the average human test taker score of 71% (~142/200 questions). Claude correctly answered 180 (89%) questions, a score greater than that of human test takers and significantly better than GPT-4o ( P< .001). Claude scored significantly higher than GPT-4o in almost all question categories. GPT-4o provided different responses to identically worded trials for 27 (21%) of 130 general questions and 3 (4%) of 70 orthopedic questions ( P= .002), while Claude did not have a significant difference in variability between these 2 groups (general: 16/130, 12% vs orthopedic: 3/70, 4%; P =.06). Neither GPT-4o or Claude showed a significant difference in the rate of most common incorrect response selection between question categories. Claude showed no significant difference in the rate of most common incorrect response selection between question categories.
Conclusions
These results suggest that GPT-4o can potentially answer both general and specialty-specific ethical questions with similar proficiency to sample groups of both medical students and orthopedic residents, while Claude AI performs significantly better than both humans and GPT-4o. Variables such as AI model framework and training data may drive the observed difference in performance, but the exact cause cannot be definitively isolated without intentional testing. Therefore, further research is needed to ensure safety by minimizing output variability before integrating AI as a patient-facing resource.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Hand Surgery, Department of Orthopaedic Surgery, Emory University, 80 Jesse Hill Jr Drive SE, Atlanta, GA, 30303, United States, 1 404-778-1550.
ORCID-Profile: Karishma R Desai, Anna L Gorsky, Nicole A Zelenski
Zitieren
APA
Desai, K. R., Gorsky, A. L., & Zelenski, N. A. (2026). Performance of GPT-4o and Claude in Medical Ethics Scenarios: Comparative Study. JMIR Medical Education, 12, e70199-e70199. https://doi.org/10.2196/70199
Vancouver
Desai KR, Gorsky AL, Zelenski NA. Performance of GPT-4o and Claude in Medical Ethics Scenarios: Comparative Study. JMIR Medical Education. 2026;12:e70199-e70199. doi:10.2196/70199
BibTeX
@article{Desai2026,
author = {Desai, Karishma R and Gorsky, Anna L and Zelenski, Nicole A},
title = {Performance of GPT-4o and Claude in Medical Ethics Scenarios: Comparative Study},
journal = {JMIR Medical Education},
year = {2026},
volume = {12},
pages = {e70199--e70199},
doi = {10.2196/70199}
}
EUVIMED