EUVIMEDEUVIMED entdecken
Crossref

Evaluating the Performance of GPT ‐4.0 in Generating and Refining Multiple‐Choice Questions for Dental Education

Jiashun Wu iD, Huanzhong Ji, Yanqing Xie, Shijun Kuang iD, Tao Yang iD

European Journal of Dental Education · 17.06.2026

DOI: 10.1111/eje.70217 ↗

Kurz gefasst

Fachartikel in der Zeitschrift European Journal of Dental Education, online erschienen am 17.06.2026. Verfasst von Jiashun Wu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Hospital of Stomatology, Guanghua School of Stomatology, Guangdong Provincial Key Laboratory of Stomatology, Sun Yat-Sen University, Guangzhou, China. Die Arbeit nennt 28 Literaturangaben. Gefördert von: Natural Science Foundation of Guangdong Province.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

The integration of large‐language models into medical education assessment holds transformative potential, yet rigorous evaluation of their capabilities in generating specialized examination content remains underexplored. This study aims to explore the performance of GPT‐4.0 in crafting multiple‐choice questions (MCQs) within the domain of oral and maxillofacial surgery (OMFS), benchmarking it against GPT‐3.5 and GPT‐4o, and evaluates its capacity to assess and revise defective items.

Methods

100 MCQs generated by GPT‐4.0 were evaluated on six aspects with 5‐point scales. Comparative analyses were conducted among ChatGPT 4.0, ChatGPT 3.5, and ChatGPT 4o. Additionally, 43 defective MCQs generated by GPTs were input into GPT‐4.0 to assess its capabilities in detecting and providing modification suggestions. Inter‐rater reliability between the two expert reviewers was assessed using Cohen's kappa coefficient.

Results

GPT‐4.0 generated questions rapidly and efficiently, with 87% of them being of high quality. However, 6% required replacement due to issues such as interdisciplinary ambiguity, flawed distractors, or item duplication, while an additional 7% needed minor revisions before use. The comparative analysis highlighted GPT‐4.0's marked advantage over GPT‐3.5, which exhibited higher defect rates and disciplinary misalignments. Notably, GPT‐4.0's ability to detect and revise defective questions proved limited, excelling only in identifying duplicate items but faltering in recognizing disciplinary overlaps or erroneous answers.

Conclusion

GPT‐4.0 proves to be a highly capable tool for generating high‐quality OMFS MCQs, outperforming both GPT‐3.5 and GPT‐4o. Nevertheless, its limited capacity to detect nuanced medical flaws and distinguish between closely related disciplines necessitates mandatory human expert oversight to ensure the accuracy and validity of medical assessments.

Schlagwörter

Stichwörter der Autor:innen

GPT‐4.0artificial intelligenceautomated question generationlarge‐language modelsmedical assessmentmultiple‐choice questions

Autor:innen und Institutionen

  • Hospital of Stomatology, Guanghua School of Stomatology, Guangdong Provincial Key Laboratory of Stomatology, Sun Yat-Sen University, Guangzhou, China.

ORCID-Profile: Jiashun Wu, Shijun Kuang, Tao Yang

Förderung

  • Natural Science Foundation of Guangdong Province

Zitieren

APA

Wu, J., Ji, H., Xie, Y., Kuang, S., & Yang, T. (2026). Evaluating the Performance of GPT ‐4.0 in Generating and Refining Multiple‐Choice Questions for Dental Education. European Journal of Dental Education. https://doi.org/10.1111/eje.70217

Vancouver

Wu J, Ji H, Xie Y, Kuang S, Yang T. Evaluating the Performance of GPT ‐4.0 in Generating and Refining Multiple‐Choice Questions for Dental Education. European Journal of Dental Education. 2026. doi:10.1111/eje.70217

BibTeX
@article{Wu2026,
  author = {Wu, Jiashun and Ji, Huanzhong and Xie, Yanqing and Kuang, Shijun and Yang, Tao},
  title = {Evaluating the Performance of GPT ‐4.0 in Generating and Refining Multiple‐Choice Questions for Dental Education},
  journal = {European Journal of Dental Education},
  year = {2026},
  doi = {10.1111/eje.70217}
}
Kostenlos starten →