Learning to Reuse Distractors to Support Multiple-Choice Question Generation in Education
IEEE Transactions on Learning Technologies · 2024 · 17 · 375-390
DOI: 10.1109/tlt.2022.3226523 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Learning Technologies, 2024. Verfasst von Semere Kiros Bitew und 5 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: Flanders Innovation and Entrepreneurship, Flanders, Belgium, AI-Driven e-Assessment, Vlaamse regering.
Abstract (Originalsprache)
Multiple-choice questions (MCQs) are widely used in digital learning systems, as they allow for automating the assessment process. However, owing to the increased digital literacy of students and the advent of social media platforms, MCQ tests are widely shared online, and teachers are continuously challenged to create new questions, which is an expensive and time-consuming task. A particularly sensitive aspect of MCQ creation is to devise relevant distractors, i.e., wrong answers that are not easily identifiable as being wrong. This article studies how a large existing set of manually created answers and distractors for questions over a variety of domains, subjects, and languages can be leveraged to help teachers in creating new MCQs, by the smart reuse of existing distractors. We built several data-driven models based on context-aware question and distractor representations and compared them with static feature-based models. The proposed models are evaluated with automated metrics and in a realistic user test with teachers. Both automatic and human evaluations indicate that context-aware models consistently outperform a static feature-based approach. For our best-performing context-aware model, on average, three distractors out of the ten shown to teachers were rated as high-quality distractors. We create a performance benchmark, and make it public, to enable comparison between different approaches and to introduce a more standardized evaluation of the task. The benchmark contains a test of 298 educational questions covering multiple subjects and languages and a 77k multilingual pool of distractor vocabulary for future research.
Förderung
- Flanders Innovation and Entrepreneurship, Flanders, Belgium
- AI-Driven e-Assessment
- Vlaamse regering
- Onderzoeksprogramma Artificile Intelligentie Vlaanderen program
Zitieren
APA
Bitew, S. K., Hadifar, A., Sterckx, L., Deleu, J., Develder, C., & Demeester, T. (2024). Learning to Reuse Distractors to Support Multiple-Choice Question Generation in Education. IEEE Transactions on Learning Technologies, 17, 375-390. https://doi.org/10.1109/tlt.2022.3226523
Vancouver
Bitew SK, Hadifar A, Sterckx L, Deleu J, Develder C, Demeester T. Learning to Reuse Distractors to Support Multiple-Choice Question Generation in Education. IEEE Transactions on Learning Technologies. 2024;17:375-390. doi:10.1109/tlt.2022.3226523
BibTeX
@article{Bitew2024,
author = {Bitew, Semere Kiros and Hadifar, Amir and Sterckx, Lucas and Deleu, Johannes and Develder, Chris and Demeester, Thomas},
title = {Learning to Reuse Distractors to Support Multiple-Choice Question Generation in Education},
journal = {IEEE Transactions on Learning Technologies},
year = {2024},
volume = {17},
pages = {375--390},
doi = {10.1109/tlt.2022.3226523}
}
EUVIMED