Comparative performance of artificial intelligence models on the Peruvian CODIRO 2023 dental residency examination
Journal of Dentistry and Multidisciplinary Sciences · 2026
DOI: 10.3934/jdms.2026007 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Dentistry and Multidisciplinary Sciences, 2026. Verfasst von Edgar S. Povis-Flores und 5 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben.
Abstract (Originalsprache)
Artificial intelligence based on large language models has gained increasing relevance in educational and healthcare settings due to its ability to process information and generate coherent natural language responses. In the health sciences, several studies have evaluated the performance of these systems in academic and professional licensing examinations; however, evidence regarding their performance in dentistry, particularly in Latin American contexts, remains limited. This study aimed to compare the response accuracy of four artificial intelligence language models in answering the 2023 CODIRO Dental Residency admission examination in Peru, using the official answer key as the reference standard. An observational, cross-sectional, comparative study was conducted using the complete set of 100 multiple-choice questions from the examination. Claude, Copilot, Duck IA, and Qwen were each administered the same standardized prompt for every question, and responses were classified as correct or incorrect against the official key. Because all four models answered the same questions, accuracy was compared using Cochran's Q test, with pairwise exact McNemar tests and Holm correction applied when the omnibus test was significant. Copilot achieved the highest accuracy (100%), followed by Claude and Qwen (96% each) and Duck IA (86%); Cochran's Q test indicated a significant overall difference (p < 0.001). Duck IA performed significantly worse than the other three models, without significant differences among them. Agreement among models was highest in clinical disciplines and lowest in foundational disciplines such as anatomy and histology/embryology. Among items answered incorrectly, failing models frequently selected the same incorrect option, suggesting shared misconceptions rather than independent errors. These findings suggest that large language models, including platforms with limited representation in the dental education literature, can achieve high accuracy on dental residency examination questions; however, this reflects knowledge recognition rather than clinical competence, and supports their potential only as complementary tools in dental education.
Zitieren
APA
Povis-Flores, E. S., Ramírez-Benavides, M. A., Meza-Ponce, F., Ramirez-Acuña, A. G., Benavides, M. L. R., & Saravia-Rojas, M. A. (2026). Comparative performance of artificial intelligence models on the Peruvian CODIRO 2023 dental residency examination. Journal of Dentistry and Multidisciplinary Sciences. https://doi.org/10.3934/jdms.2026007
Vancouver
Povis-Flores ES, Ramírez-Benavides MA, Meza-Ponce F, Ramirez-Acuña AG, Benavides MLR, Saravia-Rojas MA. Comparative performance of artificial intelligence models on the Peruvian CODIRO 2023 dental residency examination. Journal of Dentistry and Multidisciplinary Sciences. 2026. doi:10.3934/jdms.2026007
BibTeX
@article{PovisFlores2026,
author = {Povis-Flores, Edgar S. and Ramírez-Benavides, Mariana A. and Meza-Ponce, Fabricio and Ramirez-Acuña, Andrea G. and Benavides, Mónica L. Ramírez and Saravia-Rojas, Miguel A.},
title = {Comparative performance of artificial intelligence models on the Peruvian CODIRO 2023 dental residency examination},
journal = {Journal of Dentistry and Multidisciplinary Sciences},
year = {2026},
doi = {10.3934/jdms.2026007}
}
EUVIMED