Performance of Large Language Models in Medical Exams: A Comparison Between ChatGPT and Medical Students
Tıp Eğitimi Dünyası · 22.12.2025 · 24 (74) · 135-143
DOI: 10.25282/ted.1729174 ↗Kurz gefasst
Fachartikel in der Zeitschrift Tıp Eğitimi Dünyası, online erschienen am 22.12.2025. Verfasst von Ahmet Ozan Kaleci und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 25 Literaturangaben.
Abstract (Originalsprache)
Background
Medical education in Türkiye is delivered through a six-year, discipline-based curriculum aligned with global trends. The assessment process largely relies on multiple-choice questions, placing a significant preparation burden on faculty members. AI-powered large language models like ChatGPT have the potential to ease exam preparation, enhance feedback quality, and support personalized learning. The aim of this study is to evaluate the success of the ChatGPT-4o model performs when answering multiple-choice (MCQ) questions on medical education exams. Additionally, by comparing exam performance and consistency to student success, we explore the potential benefits of AI-supported models to medical education.
Methods
This cross-sectional, analytical investigation was carried out in Türkiye at the [XX] University Faculty of Medicine. During the 2023–2024 academic year, ChatGPT solved multiple-choice questions from seven board exams and one final exam for third-year students, the results were compared with the students' achievements. Statistical analysis included descriptive statistics, correlation analyses, chi-square tests, McNemar tests, and t-tests for independent samples.
Results
With a 90.2% correct response percentage, ChatGPT outperformed the entire class, outperforming 293 other students. There was no significant difference in the correct response rate between the surgical, internal, and fundamental medical sciences (p = 0.742). In several fields, such psychiatry, neurology, and medical genetics, 100% success was attained. Forensic medicine, family medicine, medical ethics, pulmonary medicine, and thoracic surgery all had success rates that were lower than 80%. A retest conducted two months later revealed that ChatGPT's success rate had somewhat risen, with response consistency standing at 91.4%.
Conclusions
With a high success rate on medical education tests, ChatGPT has shown a great deal of promise to help both students and instructors. The integration of AI models into educational systems should be done strategically and with a human-centered approach, though, given the constraints in areas like clinical reasoning, ethical evaluation, and human-centered medical education. It is important to design instructional strategies in the future that combine artificial intelligence technologies with human skills.
Zitieren
APA
Kaleci, A. O., Şahinbaş, B., Ağadayı, E., Çelikkaya, S. İ., Altun, A., & Kardan, E. K. (2025). Performance of Large Language Models in Medical Exams: A Comparison Between ChatGPT and Medical Students. Tıp Eğitimi Dünyası, 24(74), 135-143. https://doi.org/10.25282/ted.1729174
Vancouver
Kaleci AO, Şahinbaş B, Ağadayı E, Çelikkaya Sİ, Altun A, Kardan EK. Performance of Large Language Models in Medical Exams: A Comparison Between ChatGPT and Medical Students. Tıp Eğitimi Dünyası. 2025;24(74):135-143. doi:10.25282/ted.1729174
BibTeX
@article{Kaleci2025,
author = {Kaleci, Ahmet Ozan and Şahinbaş, Burcu and Ağadayı, Ezgi and Çelikkaya, Sümeyye İdil and Altun, Ahmet and Kardan, Emre Kemal},
title = {Performance of Large Language Models in Medical Exams: A Comparison Between ChatGPT and Medical Students},
journal = {Tıp Eğitimi Dünyası},
year = {2025},
volume = {24},
number = {74},
pages = {135--143},
doi = {10.25282/ted.1729174}
}
EUVIMED