EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Misconception-Aware Prompting Can Generate Authentic Analytical Multiple-Choice Questions

Machi Shimmei, Masaki Uto, Yuichiroh Matsubayashi, Kentaro Inui, Aditi Mallavarapu, Noboru Matsuda

IEEE Access · 2026 · 14 · 53513-53528

DOI: 10.1109/access.2026.3681235 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Machi Shimmei und 5 weiteren Autor:innen. Die Arbeit nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, North Carolina State University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Although automatic multiple-choice question (MCQ) generation has been extensively studied, generating MCQs that require deep reasoning remains underexplored. This study aims to develop a framework for automatically constructing Analytical Multiple-Choice Questions (Analytical MCQs) using a pre-trained large language model. Analytical MCQs present sentence-level assertions as choice items and are designed to evaluate students’ higher-order reasoning about complex subject matter. One of the main challenges in this task lies in generating plausible yet incorrect assertions (i.e., distractors) that reflect students’ misunderstandings. To address this challenge, we utilize students’ written responses to open-ended questions, which offer insights into their conceptual understanding of a topic. Building on this foundation, we introduce AnaQuest, a prompting technique that integrates these open-ended responses into the generation of Analytical MCQs. AnaQuest analyzes these responses, identifies common errors, and generates incorrect assertions alongside correct ones. We evaluated the quality of AI-generated assertions by comparing their Item Response Theory (IRT) parameters, collected from exams where the generated questions were used, with those of certified human-crafted assertions. The analysis showed that assertions generated by AnaQuest more closely resembled human-crafted ones in both discrimination power and difficulty level, compared to those generated by a baseline ChatGPT model that does not incorporate student misunderstandings. To our knowledge, this is the first study to demonstrate the importance of using knowledge of students’ misconceptions to generate high-quality Multiple-Choice questions.

Förderung

  • National Science Foundation, North Carolina State University

Zitieren

APA

Shimmei, M., Uto, M., Matsubayashi, Y., Inui, K., Mallavarapu, A., & Matsuda, N. (2026). Misconception-Aware Prompting Can Generate Authentic Analytical Multiple-Choice Questions. IEEE Access, 14, 53513-53528. https://doi.org/10.1109/access.2026.3681235

Vancouver

Shimmei M, Uto M, Matsubayashi Y, Inui K, Mallavarapu A, Matsuda N. Misconception-Aware Prompting Can Generate Authentic Analytical Multiple-Choice Questions. IEEE Access. 2026;14:53513-53528. doi:10.1109/access.2026.3681235

BibTeX
@article{Shimmei2026,
  author = {Shimmei, Machi and Uto, Masaki and Matsubayashi, Yuichiroh and Inui, Kentaro and Mallavarapu, Aditi and Matsuda, Noboru},
  title = {Misconception-Aware Prompting Can Generate Authentic Analytical Multiple-Choice Questions},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {53513--53528},
  doi = {10.1109/access.2026.3681235}
}
Kostenlos starten →