EUVIMEDEUVIMED entdecken
Crossref

Performance of Large Language Models in Medical Exams: A Comparison Between ChatGPT and Medical Students

Ahmet Ozan Kaleci, Burcu Şahinbaş, Ezgi Ağadayı, Sümeyye İdil Çelikkaya, Ahmet Altun, Emre Kemal Kardan

Tıp Eğitimi Dünyası · 22.12.2025 · 24 (74) · 135-143

DOI: 10.25282/ted.1729174 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Tıp Eğitimi Dünyası, online erschienen am 22.12.2025. Verfasst von Ahmet Ozan Kaleci und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 25 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Medical education in Türkiye is delivered through a six-year, discipline-based curriculum aligned with global trends. The assessment process largely relies on multiple-choice questions, placing a significant preparation burden on faculty members. AI-powered large language models like ChatGPT have the potential to ease exam preparation, enhance feedback quality, and support personalized learning. The aim of this study is to evaluate the success of the ChatGPT-4o model performs when answering multiple-choice (MCQ) questions on medical education exams. Additionally, by comparing exam performance and consistency to student success, we explore the potential benefits of AI-supported models to medical education.

Methods

This cross-sectional, analytical investigation was carried out in Türkiye at the [XX] University Faculty of Medicine. During the 2023–2024 academic year, ChatGPT solved multiple-choice questions from seven board exams and one final exam for third-year students, the results were compared with the students' achievements. Statistical analysis included descriptive statistics, correlation analyses, chi-square tests, McNemar tests, and t-tests for independent samples.

Results

With a 90.2% correct response percentage, ChatGPT outperformed the entire class, outperforming 293 other students. There was no significant difference in the correct response rate between the surgical, internal, and fundamental medical sciences (p = 0.742). In several fields, such psychiatry, neurology, and medical genetics, 100% success was attained. Forensic medicine, family medicine, medical ethics, pulmonary medicine, and thoracic surgery all had success rates that were lower than 80%. A retest conducted two months later revealed that ChatGPT's success rate had somewhat risen, with response consistency standing at 91.4%.

Conclusions

With a high success rate on medical education tests, ChatGPT has shown a great deal of promise to help both students and instructors. The integration of AI models into educational systems should be done strategically and with a human-centered approach, though, given the constraints in areas like clinical reasoning, ethical evaluation, and human-centered medical education. It is important to design instructional strategies in the future that combine artificial intelligence technologies with human skills.

Zitieren

APA

Kaleci, A. O., Şahinbaş, B., Ağadayı, E., Çelikkaya, S. İ., Altun, A., & Kardan, E. K. (2025). Performance of Large Language Models in Medical Exams: A Comparison Between ChatGPT and Medical Students. Tıp Eğitimi Dünyası, 24(74), 135-143. https://doi.org/10.25282/ted.1729174

Vancouver

Kaleci AO, Şahinbaş B, Ağadayı E, Çelikkaya Sİ, Altun A, Kardan EK. Performance of Large Language Models in Medical Exams: A Comparison Between ChatGPT and Medical Students. Tıp Eğitimi Dünyası. 2025;24(74):135-143. doi:10.25282/ted.1729174

BibTeX
@article{Kaleci2025,
  author = {Kaleci, Ahmet Ozan and Şahinbaş, Burcu and Ağadayı, Ezgi and Çelikkaya, Sümeyye İdil and Altun, Ahmet and Kardan, Emre Kemal},
  title = {Performance of Large Language Models in Medical Exams: A Comparison Between ChatGPT and Medical Students},
  journal = {Tıp Eğitimi Dünyası},
  year = {2025},
  volume = {24},
  number = {74},
  pages = {135--143},
  doi = {10.25282/ted.1729174}
}
Kostenlos starten →