Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation Using Knowledge Graphs and Large Language Models
Machine Learning and Knowledge Extraction · 20.05.2026 · 8 (5) · 137
DOI: 10.3390/make8050137 ↗Kurz gefasst
Fachartikel in der Zeitschrift Machine Learning and Knowledge Extraction, online erschienen am 20.05.2026. Verfasst von Mehmet Can Şakiroğlu und 2 weiteren Autor:innen. Die Arbeit nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Union.
Abstract (Originalsprache)
Generating multiple-choice questions (MCQs) with difficulty estimation remains challenging in automated MCQ-generation systems used in adaptive, AI-assisted education. This study proposes a novel methodology for generating MCQs with difficulty estimation from input documents by utilizing knowledge graphs (KGs) and large language models (LLMs). Our approach uses an LLM to construct a KG from input documents, from which MCQs are then systematically generated. Each MCQ is generated by selecting a node from the KG as the key, sampling a related triple or quintuple—optionally augmented with an extra triple—and prompting an LLM to generate a corresponding stem from these graph components. Distractors are then selected from the KG. For each MCQ, nine difficulty signals are computed and combined into a unified difficulty score using a data-driven approach. Within a 150-MCQ, proof-of-concept dataset from Wikipedia, the proposed signals show interpretable associations with empirical incorrect-answer rates aligning with human responses/performance. The results support the feasibility of the proposed pipeline, yet a larger-scale human study may be required to establish deployment-scale validity. Our approach improves automated MCQ generation by integrating structured knowledge representations with LLMs and a data-driven difficulty estimation model.
Förderung
- European Union
Zitieren
APA
Şakiroğlu, M. C., Güvenir, H. A., & Kaya, K. (2026). Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation Using Knowledge Graphs and Large Language Models. Machine Learning and Knowledge Extraction, 8(5), 137. https://doi.org/10.3390/make8050137
Vancouver
Şakiroğlu MC, Güvenir HA, Kaya K. Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation Using Knowledge Graphs and Large Language Models. Machine Learning and Knowledge Extraction. 2026;8(5):137. doi:10.3390/make8050137
BibTeX
@article{akirolu2026,
author = {Şakiroğlu, Mehmet Can and Güvenir, Halil Altay and Kaya, Kamer},
title = {Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation Using Knowledge Graphs and Large Language Models},
journal = {Machine Learning and Knowledge Extraction},
year = {2026},
volume = {8},
number = {5},
pages = {137},
doi = {10.3390/make8050137}
}
EUVIMED