EUVIMEDEUVIMED entdecken
ÜbersichtsarbeitOpen Access · CC BY 4.0

ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: a literature review

Yavuz Selim Kıyak iD, Emre Emekli

Postgraduate Medical Journal · 18.10.2024 · 100 (1189) · 858-865

DOI: 10.1093/postmj/qgae065 ↗

Kurz gefasst

Übersichtsarbeit in der Zeitschrift Postgraduate Medical Journal, online erschienen am 06.06.2024. Verfasst von Yavuz Selim Kıyak und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Education and Informatics, Faculty of Medicine, Gazi University, Ankara 06500, Turkey. Hauptthemen laut MEDLINE: Educational Measurement, Education, Medical. Die Arbeit wurde bisher 148-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

ChatGPT’s role in creating multiple-choice questions (MCQs) is growing but the validity of these artificial-intelligence-generated questions is unclear. This literature review was conducted to address the urgent need for understanding the application of ChatGPT in generating MCQs for medical education. Following the database search and screening of 1920 studies, we found 23 relevant studies. We extracted the prompts for MCQ generation and assessed the validity evidence of MCQs. The findings showed that prompts varied, including referencing specific exam styles and adopting specific personas, which align with recommended prompt engineering tactics. The validity evidence covered various domains, showing mixed accuracy rates, with some studies indicating comparable quality to human-written questions, and others highlighting differences in difficulty and discrimination levels, alongside a significant reduction in question creation time. Despite its efficiency, we highlight the necessity of careful review and suggest a need for further research to optimize the use of ChatGPT in question generation. Main messages Ensure high-quality outputs by utilizing well-designed prompts; medical educators should prioritize the use of detailed, clear ChatGPT prompts when generating MCQs. Avoid using ChatGPT-generated MCQs directly in examinations without thorough review to prevent inaccuracies and ensure relevance. Leverage ChatGPT’s potential to streamline the test development process, enhancing efficiency without compromising quality.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Educational MeasurementEducation, MedicalHumansReproducibility of Results

Stichwörter der Autor:innen

ChatGPTChatGPT promptsartificial intelligenceautomatic item generationmultiple-choice questions

Autor:innen und Institutionen

  • Department of Medical Education and Informatics, Faculty of Medicine, Gazi University, Ankara 06500, Turkey.
  • Department of Radiology, Faculty of Medicine, Eskişehir Osmangazi University, Eskişehir 26040, Turkey.

ORCID-Profile: Yavuz Selim Kıyak

Zitieren

APA

Kıyak, Y. S., & Emekli, E. (2024). ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: a literature review. Postgraduate Medical Journal, 100(1189), 858-865. https://doi.org/10.1093/postmj/qgae065

Vancouver

Kıyak YS, Emekli E. ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: a literature review. Postgraduate Medical Journal. 2024;100(1189):858-865. doi:10.1093/postmj/qgae065

BibTeX
@article{Kyak2024,
  author = {Kıyak, Yavuz Selim and Emekli, Emre},
  title = {ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: a literature review},
  journal = {Postgraduate Medical Journal},
  year = {2024},
  volume = {100},
  number = {1189},
  pages = {858--865},
  doi = {10.1093/postmj/qgae065}
}
Kostenlos starten →