EUVIMEDEUVIMED entdecken
Crossref

ChatGPT's Medical Exam Performance: Version and Language Analysis in General Surgery Fellowship Exam

Süleyman Orman

European Journal of Human Health · 2025 · 1 (1) · 1-9

DOI: 10.29228/ejhh.80681 ↗

Kurz gefasst

Fachartikel in der Zeitschrift European Journal of Human Health, 2025. Verfasst von Süleyman Orman. Die Arbeit wurde bisher 2-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

The integration of Artificial Intelligence (AI) in medical education has the potential to revolutionize learning and assessment.This study evaluates the performance of ChatGPT-3.5 and ChatGPT-4 on the General Surgery Fellowship Examination (GSFE) in Turkey, comparing their accuracy in answering multiple-choice questions (MCQs) in Turkish and English.

Methods

255 retired and publicly available GSFE questions (2011-2022) were analyzed.Questions were first presented in Turkish and subsequently translated into English for re-evaluation.ChatGPT-3.5 and ChatGPT-4 were prompted as if they were general surgeons answering the MCQs.The accuracy of responses was assessed, and statistical analyses were performed to identify significant differences between the bots and languages.

Results

In Turkish, ChatGPT-3.5 achieved 66.66% accuracy (170/255 correct answers), while ChatGPT-4 scored 69.41% (177/255).In English, ChatGPT-3.5 achieved 67.05% accuracy (171/255), and ChatGPT-4 scored 70.19% (179/255).Statistically significant differences were observed between ChatGPT-3.5 and ChatGPT-4 for both Turkish (p 0.05).It was found that Cohen's Kappa number was quite high between pairwise comparisons between applications, and the consistency among the four applications showed similarly high agreement (Fleiss' Kappa = 0.95, p

Zitieren

APA

Orman, S. (2025). ChatGPT's Medical Exam Performance: Version and Language Analysis in General Surgery Fellowship Exam. European Journal of Human Health, 1(1), 1-9. https://doi.org/10.29228/ejhh.80681

Vancouver

Orman S. ChatGPT's Medical Exam Performance: Version and Language Analysis in General Surgery Fellowship Exam. European Journal of Human Health. 2025;1(1):1-9. doi:10.29228/ejhh.80681

BibTeX
@article{Orman2025,
  author = {Orman, Süleyman},
  title = {ChatGPT's Medical Exam Performance: Version and Language Analysis in General Surgery Fellowship Exam},
  journal = {European Journal of Human Health},
  year = {2025},
  volume = {1},
  number = {1},
  pages = {1--9},
  doi = {10.29228/ejhh.80681}
}
Kostenlos starten →