EUVIMEDEUVIMED entdecken
Systematische ÜbersichtOpen Access · CC BY-NC 4.0

Validity of AI-generated multiple-choice questions in medical education: a systematic review

Yavuz Selim Kıyak iD, Abdullah Bedir Kaya iD, Emre Emekli

Postgraduate Medical Journal · 20.05.2026

DOI: 10.1093/postmj/qgag057 ↗

Kurz gefasst

Systematische Übersicht in der Zeitschrift Postgraduate Medical Journal, online erschienen am 20.05.2026. Verfasst von Yavuz Selim Kıyak und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Education and Informatics, Faculty of Medicine, Gazi University, Ankara, Turkiye. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 77 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) are increasingly used to generate multiple-choice questions (MCQs) in medical education. We conducted a systematic review following PRISMA 2020, searching PubMed, Web of Science, Scopus, and ERIC through 15 February 2026. Two reviewers independently screened 1352 records, extracted data and assessed methodological quality using the Joanna Briggs Institute checklist. Findings were synthesized according to Messick’s five sources of validity evidence. Seventy-one studies from 24 countries were included. Most relied on expert review rather than learner-based testing. All studies reported content evidence, whereas fewer addressed relations to other variables (40/71), response process (35/71), internal structure (31/71) or consequences (25/71). Error rates ranged from <1% to 45%. Median item difficulty was 0.67 and discrimination 0.28, with reliability between 0.51–0.81. Studies reported substantial efficiency gains, including up to 31-fold time savings. LLMs appear useful drafting tools but current evidence does not yet support unsupervised use in summative assessment.

Schlagwörter

Stichwörter der Autor:innen

artificial intelligencelarge language modelsmedical educationmultiple-choice questionsvalidity

Autor:innen und Institutionen

  • Department of Medical Education and Informatics, Faculty of Medicine, Gazi University, Ankara, Turkiye.
  • Department of Computer Technologies, Hitit University, Çorum, Turkiye.
  • Department of Radiology, Faculty of Medicine, Eskişehir Osmangazi University, Eskişehir, Turkiye.

ORCID-Profile: Yavuz Selim Kıyak, Abdullah Bedir Kaya

Zitieren

APA

Kıyak, Y. S., Kaya, A. B., & Emekli, E. (2026). Validity of AI-generated multiple-choice questions in medical education: a systematic review. Postgraduate Medical Journal. https://doi.org/10.1093/postmj/qgag057

Vancouver

Kıyak YS, Kaya AB, Emekli E. Validity of AI-generated multiple-choice questions in medical education: a systematic review. Postgraduate Medical Journal. 2026. doi:10.1093/postmj/qgag057

BibTeX
@article{Kyak2026,
  author = {Kıyak, Yavuz Selim and Kaya, Abdullah Bedir and Emekli, Emre},
  title = {Validity of AI-generated multiple-choice questions in medical education: a systematic review},
  journal = {Postgraduate Medical Journal},
  year = {2026},
  doi = {10.1093/postmj/qgag057}
}
Kostenlos starten →