Evaluation of the performance of artificial intelligence platforms in answering and generating new questions in prosthetic dentistry specialization
Anatolian Current Medical Journal · 22.05.2026 · 8 (3) · 409-416
DOI: 10.38053/acmj.1848512 ↗Kurz gefasst
Fachartikel in der Zeitschrift Anatolian Current Medical Journal, online erschienen am 22.05.2026. Verfasst von Aliye İpek Kuşçu und 2 weiteren Autor:innen. Die Arbeit nennt 41 Literaturangaben. Gefördert von: There is no Supporting Institution.
Abstract (Originalsprache)
Aims
This study aimed to evaluate large language models (LLMs) not only in answering Dentistry Specialty Examination (DUS) questions but also in generating new DUS-format questions, with expert validation of educational and clinical quality.
Methods
A total of 130 official DUS questions published between 2012 and 2021 were used to assess answering performance of four LLMs (ChatGPT, Gemini, DeepSeek, and Grok). Additionally, each model generated 20 new multiple-choice questions (n=80), which were independently evaluated by expert prosthodontists for content accuracy, clinical relevance, discriminative capacity, and conformity with DUS standards. Expert-approved questions were subsequently re-answered by all models to enable cross-model performance analysis. Model performances were compared using descriptive statistics, one-sample proportion tests against chance level (p₀=0.20), and inter-model comparisons using Cochran’s Q and McNemar tests.
Results
ChatGPT achieved the highest overall accuracy on historical DUS questions (81.3%), followed by Gemini and DeepSeek (72.8% and 70.3%) and Grok (68.8%). In expert-validated AI-generated questions, overall accuracy rates ranged between 71.3% and 78.8% across models, with no statistically significant inter-model difference (Q=3.82, p=0.28). All models performed significantly above chance level (p
Förderung
- There is no Supporting Institution
Zitieren
APA
Kuşçu, A. İ., Çınarer, G., & Kuşçu, S. (2026). Evaluation of the performance of artificial intelligence platforms in answering and generating new questions in prosthetic dentistry specialization. Anatolian Current Medical Journal, 8(3), 409-416. https://doi.org/10.38053/acmj.1848512
Vancouver
Kuşçu Aİ, Çınarer G, Kuşçu S. Evaluation of the performance of artificial intelligence platforms in answering and generating new questions in prosthetic dentistry specialization. Anatolian Current Medical Journal. 2026;8(3):409-416. doi:10.38053/acmj.1848512
BibTeX
@article{Kuu2026,
author = {Kuşçu, Aliye İpek and Çınarer, Gökalp and Kuşçu, Süha},
title = {Evaluation of the performance of artificial intelligence platforms in answering and generating new questions in prosthetic dentistry specialization},
journal = {Anatolian Current Medical Journal},
year = {2026},
volume = {8},
number = {3},
pages = {409--416},
doi = {10.38053/acmj.1848512}
}
EUVIMED