EUVIMEDEUVIMED entdecken
Crossref

Evaluation of GPT-4o and GPT o1 pro in answering Turkish Neurosurgical Society proficiency board exam questions: A comparative study

Alaeddin Acar, Eray Tekirdaş

Pamukkale Medical Journal · 10.07.2026 · 19 (3) · 488-496

DOI: 10.31362/patd.1735217 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Pamukkale Medical Journal, online erschienen am 10.07.2026. Verfasst von Alaeddin Acar und 1 weiteren Autor:innen. Die Arbeit nennt 22 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

This study aims to evaluate and compare the answering capabilities and accuracy of GPT-4o and GPT o1 pro for a set of 98 questions, including image-based questions, from the 2024 neurosurgery board certification examination.Materials and methods: Answers to 98 questions, including image-based questions, were generated using GPT-4o and GPT o1 pro. We evaluated their performance through both one-shot and multiple-attempt methods, calculating accuracy and consistency rates. Comparisons were made between text-based and image-based questions to assess multimodal reasoning.

Results

GPT o1 pro achieved a one-shot accuracy of 92.9%, while GPT-4o reached 71.4% (p<0.001). Over three attempts, GPT o1 pro maintained 91.8% accuracy compared to GPT-4o’s 72.4% (p<0.001). GPT o1 pro also showed higher consistency (91.8% vs 78.6%, p=0.009). For text-only questions, GPT o1 pro reached 96.3% one-shot accuracy, compared to GPT-4o’s 72.8%. For image-based questions, GPT o1 pro scored 76.5%, and GPT-4o scored 64.7%, though this difference was not statistically significant (p=0.68). These results suggest that GPT o1 pro outperforms GPT-4o, especially for text-based items, but both models need further improvement for image-based tasks.

Conclusion

Both GPT o1 pro and GPT-4o successfully passed the 2024 Turkish Neurosurgical Society Board Certification Examination. GPT o1 pro showed notably higher accuracy and consistency than GPT-4o, suggesting a promising role for chain-of-thought reasoning in medical education.

Zitieren

APA

Acar, A., & Tekirdaş, E. (2026). Evaluation of GPT-4o and GPT o1 pro in answering Turkish Neurosurgical Society proficiency board exam questions: A comparative study. Pamukkale Medical Journal, 19(3), 488-496. https://doi.org/10.31362/patd.1735217

Vancouver

Acar A, Tekirdaş E. Evaluation of GPT-4o and GPT o1 pro in answering Turkish Neurosurgical Society proficiency board exam questions: A comparative study. Pamukkale Medical Journal. 2026;19(3):488-496. doi:10.31362/patd.1735217

BibTeX
@article{Acar2026,
  author = {Acar, Alaeddin and Tekirdaş, Eray},
  title = {Evaluation of GPT-4o and GPT o1 pro in answering Turkish Neurosurgical Society proficiency board exam questions: A comparative study},
  journal = {Pamukkale Medical Journal},
  year = {2026},
  volume = {19},
  number = {3},
  pages = {488--496},
  doi = {10.31362/patd.1735217}
}
Kostenlos starten →