Evaluating the Performance of GPT ‐4.0 in Generating and Refining Multiple‐Choice Questions for Dental Education
European Journal of Dental Education · 17.06.2026
DOI: 10.1111/eje.70217 ↗Kurz gefasst
Fachartikel in der Zeitschrift European Journal of Dental Education, online erschienen am 17.06.2026. Verfasst von Jiashun Wu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Hospital of Stomatology, Guanghua School of Stomatology, Guangdong Provincial Key Laboratory of Stomatology, Sun Yat-Sen University, Guangzhou, China. Die Arbeit nennt 28 Literaturangaben. Gefördert von: Natural Science Foundation of Guangdong Province.
Abstract (Originalsprache)
Purpose
The integration of large‐language models into medical education assessment holds transformative potential, yet rigorous evaluation of their capabilities in generating specialized examination content remains underexplored. This study aims to explore the performance of GPT‐4.0 in crafting multiple‐choice questions (MCQs) within the domain of oral and maxillofacial surgery (OMFS), benchmarking it against GPT‐3.5 and GPT‐4o, and evaluates its capacity to assess and revise defective items.
Methods
100 MCQs generated by GPT‐4.0 were evaluated on six aspects with 5‐point scales. Comparative analyses were conducted among ChatGPT 4.0, ChatGPT 3.5, and ChatGPT 4o. Additionally, 43 defective MCQs generated by GPTs were input into GPT‐4.0 to assess its capabilities in detecting and providing modification suggestions. Inter‐rater reliability between the two expert reviewers was assessed using Cohen's kappa coefficient.
Results
GPT‐4.0 generated questions rapidly and efficiently, with 87% of them being of high quality. However, 6% required replacement due to issues such as interdisciplinary ambiguity, flawed distractors, or item duplication, while an additional 7% needed minor revisions before use. The comparative analysis highlighted GPT‐4.0's marked advantage over GPT‐3.5, which exhibited higher defect rates and disciplinary misalignments. Notably, GPT‐4.0's ability to detect and revise defective questions proved limited, excelling only in identifying duplicate items but faltering in recognizing disciplinary overlaps or erroneous answers.
Conclusion
GPT‐4.0 proves to be a highly capable tool for generating high‐quality OMFS MCQs, outperforming both GPT‐3.5 and GPT‐4o. Nevertheless, its limited capacity to detect nuanced medical flaws and distinguish between closely related disciplines necessitates mandatory human expert oversight to ensure the accuracy and validity of medical assessments.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Hospital of Stomatology, Guanghua School of Stomatology, Guangdong Provincial Key Laboratory of Stomatology, Sun Yat-Sen University, Guangzhou, China.
ORCID-Profile: Jiashun Wu, Shijun Kuang, Tao Yang
Förderung
- Natural Science Foundation of Guangdong Province
Zitieren
APA
Wu, J., Ji, H., Xie, Y., Kuang, S., & Yang, T. (2026). Evaluating the Performance of GPT ‐4.0 in Generating and Refining Multiple‐Choice Questions for Dental Education. European Journal of Dental Education. https://doi.org/10.1111/eje.70217
Vancouver
Wu J, Ji H, Xie Y, Kuang S, Yang T. Evaluating the Performance of GPT ‐4.0 in Generating and Refining Multiple‐Choice Questions for Dental Education. European Journal of Dental Education. 2026. doi:10.1111/eje.70217
BibTeX
@article{Wu2026,
author = {Wu, Jiashun and Ji, Huanzhong and Xie, Yanqing and Kuang, Shijun and Yang, Tao},
title = {Evaluating the Performance of GPT ‐4.0 in Generating and Refining Multiple‐Choice Questions for Dental Education},
journal = {European Journal of Dental Education},
year = {2026},
doi = {10.1111/eje.70217}
}
EUVIMED