EUVIMEDEUVIMED entdecken
Crossref

Evaluating Large Language Models as Educational Tools: A Multi-Model Analysis of Performance on the Turkish Medical Specialty Examination (TUS) Ophthalmology Questions Özet:

Burhan Başkan, Yusuf Evcimen, Özgür Bülent Timuçin, İffet Yarımağa, Celal Sağmış

Muş Alparslan Üniversitesi Sağlık Bilimleri Dergisi · 31.07.2026 · 6 (2) · 51-65

DOI: 10.71255/maunsbd.1913060 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Muş Alparslan Üniversitesi Sağlık Bilimleri Dergisi, online erschienen am 31.07.2026. Verfasst von Burhan Başkan und 4 weiteren Autor:innen. Die Arbeit nennt 29 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study aimed to comprehensively evaluate the performance of seven leading large language models (LLMs) from the 2024-2025 period on Turkish Medical Specialty Examination (TUS) ophthalmology questions, analyzing factors such as question type, chronology, and clinical area, while critically assessing the risk of data contamination and the technology's realistic potential in medical education. A set of 210 TUS ophthalmology questions (2015-2024) was presented to seven LLMs (OpenAI o1-preview, Claude 4.5 Sonnet, GPT-4o, Gemini 2.5 Pro, DeepSeek-R1, Llama 3 70B, Command R+) using a standardized protocol. Performance was statistically compared based on overall accuracy, question type (case-based vs. knowledge-based), chronological period (old: 2015-2019 vs. new: 2020-2024), and clinical subspecialty. All responses underwent blinded, expert qualitative coding for error type and confidence. OpenAI o1-preview achieved the highest overall accuracy (95.2%). Claude 4.5 Sonnet and GPT-4o demonstrated 100% accuracy on case-based questions. A significant performance gap existed between closed-source (average 91.0%) and open-source models (average 66.1%) (p

Zitieren

APA

Başkan, B., Evcimen, Y., Timuçin, Ö. B., Yarımağa, İ., & Sağmış, C. (2026). Evaluating Large Language Models as Educational Tools: A Multi-Model Analysis of Performance on the Turkish Medical Specialty Examination (TUS) Ophthalmology Questions Özet:. Muş Alparslan Üniversitesi Sağlık Bilimleri Dergisi, 6(2), 51-65. https://doi.org/10.71255/maunsbd.1913060

Vancouver

Başkan B, Evcimen Y, Timuçin ÖB, Yarımağa İ, Sağmış C. Evaluating Large Language Models as Educational Tools: A Multi-Model Analysis of Performance on the Turkish Medical Specialty Examination (TUS) Ophthalmology Questions Özet:. Muş Alparslan Üniversitesi Sağlık Bilimleri Dergisi. 2026;6(2):51-65. doi:10.71255/maunsbd.1913060

BibTeX
@article{Bakan2026,
  author = {Başkan, Burhan and Evcimen, Yusuf and Timuçin, Özgür Bülent and Yarımağa, İffet and Sağmış, Celal},
  title = {Evaluating Large Language Models as Educational Tools: A Multi-Model Analysis of Performance on the Turkish Medical Specialty Examination (TUS) Ophthalmology Questions Özet:},
  journal = {Muş Alparslan Üniversitesi Sağlık Bilimleri Dergisi},
  year = {2026},
  volume = {6},
  number = {2},
  pages = {51--65},
  doi = {10.71255/maunsbd.1913060}
}
Kostenlos starten →