Evaluation of GPT-4o and GPT o1 pro in answering Turkish Neurosurgical Society proficiency board exam questions: A comparative study
Pamukkale Medical Journal · 10.07.2026 · 19 (3) · 488-496
DOI: 10.31362/patd.1735217 ↗Kurz gefasst
Fachartikel in der Zeitschrift Pamukkale Medical Journal, online erschienen am 10.07.2026. Verfasst von Alaeddin Acar und 1 weiteren Autor:innen. Die Arbeit nennt 22 Literaturangaben.
Abstract (Originalsprache)
Purpose
This study aims to evaluate and compare the answering capabilities and accuracy of GPT-4o and GPT o1 pro for a set of 98 questions, including image-based questions, from the 2024 neurosurgery board certification examination.Materials and methods: Answers to 98 questions, including image-based questions, were generated using GPT-4o and GPT o1 pro. We evaluated their performance through both one-shot and multiple-attempt methods, calculating accuracy and consistency rates. Comparisons were made between text-based and image-based questions to assess multimodal reasoning.
Results
GPT o1 pro achieved a one-shot accuracy of 92.9%, while GPT-4o reached 71.4% (p<0.001). Over three attempts, GPT o1 pro maintained 91.8% accuracy compared to GPT-4o’s 72.4% (p<0.001). GPT o1 pro also showed higher consistency (91.8% vs 78.6%, p=0.009). For text-only questions, GPT o1 pro reached 96.3% one-shot accuracy, compared to GPT-4o’s 72.8%. For image-based questions, GPT o1 pro scored 76.5%, and GPT-4o scored 64.7%, though this difference was not statistically significant (p=0.68). These results suggest that GPT o1 pro outperforms GPT-4o, especially for text-based items, but both models need further improvement for image-based tasks.
Conclusion
Both GPT o1 pro and GPT-4o successfully passed the 2024 Turkish Neurosurgical Society Board Certification Examination. GPT o1 pro showed notably higher accuracy and consistency than GPT-4o, suggesting a promising role for chain-of-thought reasoning in medical education.
Zitieren
APA
Acar, A., & Tekirdaş, E. (2026). Evaluation of GPT-4o and GPT o1 pro in answering Turkish Neurosurgical Society proficiency board exam questions: A comparative study. Pamukkale Medical Journal, 19(3), 488-496. https://doi.org/10.31362/patd.1735217
Vancouver
Acar A, Tekirdaş E. Evaluation of GPT-4o and GPT o1 pro in answering Turkish Neurosurgical Society proficiency board exam questions: A comparative study. Pamukkale Medical Journal. 2026;19(3):488-496. doi:10.31362/patd.1735217
BibTeX
@article{Acar2026,
author = {Acar, Alaeddin and Tekirdaş, Eray},
title = {Evaluation of GPT-4o and GPT o1 pro in answering Turkish Neurosurgical Society proficiency board exam questions: A comparative study},
journal = {Pamukkale Medical Journal},
year = {2026},
volume = {19},
number = {3},
pages = {488--496},
doi = {10.31362/patd.1735217}
}
EUVIMED