A Human-AI Collaborative Assessment of AI-Generated vs. Human-Created MCQ Distractors
International Conference on Computers in Education · 01.12.2025
DOI: 10.58459/icce.2025.5941 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Conference on Computers in Education, online erschienen am 01.12.2025. Verfasst von Zifeng Liu und 4 weiteren Autor:innen.
Abstract (Originalsprache)
Multiple-choice questions (MCQs) are a widely used and effective assessment method, with the quality of distractors being crucial for their effectiveness. Recent studies have explored the use of large language models (LLMs) to generate distractors. However, generating high-quality distractors remains challenging in subjects such as computer science and mathematics that requiring strong reasoning. While some research has investigated AI-generated distractors in these fields, evaluating their quality is difficult due to existing metrics primarily focusing on surface semantics and failing to capture the necessary reasoning. This study introduces a human-AI collaborative assessment approach to evaluate distractor quality. We applied this method to compare AI-generated and human-created distractors in two high school courses: programming (349 MCQs) and statistics (576 MCQs). The findings suggest that AI-generated distractors can be competitive with human-created ones in programming courses, but significant differences exist in the understand, analyze, and evaluate types of MCQs in statistics. This study provides a practical and scalable solution for integrating and evaluating AI-generated distractors in educational assessments.
Zitieren
APA
Liu, Z., Prasad, P. G., Ngo, B., Jiao, X., & Xing, W. (2025). A Human-AI Collaborative Assessment of AI-Generated vs. Human-Created MCQ Distractors. International Conference on Computers in Education. https://doi.org/10.58459/icce.2025.5941
Vancouver
Liu Z, Prasad PG, Ngo B, Jiao X, Xing W. A Human-AI Collaborative Assessment of AI-Generated vs. Human-Created MCQ Distractors. International Conference on Computers in Education. 2025. doi:10.58459/icce.2025.5941
BibTeX
@article{Liu2025,
author = {Liu, Zifeng and Prasad, Priyadharshini Ganapathy and Ngo, Bach and Jiao, Xinyue and Xing, Wanli},
title = {A Human-AI Collaborative Assessment of AI-Generated vs. Human-Created MCQ Distractors},
journal = {International Conference on Computers in Education},
year = {2025},
doi = {10.58459/icce.2025.5941}
}
EUVIMED