Comparative Analysis of Psychometric Profiling of AI-Generated and Teacher Constructed Multiple-Choice Question Items
Journal of Innovation in Educational Assessment · 29.05.2026 · 8 (1) · 52-70
DOI: 10.66545/s7vd5d22 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Innovation in Educational Assessment, online erschienen am 29.05.2026. Verfasst von Glory Evans und 2 weiteren Autor:innen.
Abstract (Originalsprache)
The emergence of artificial intelligence (AI) in educational assessment has created new opportunities for automating test item development, yet questions remain regarding the psychometric soundness of AI-generated instruments compared to teacher-constructed tests. This study undertook a comparative analysis of the psychometric properties of multiple-choice questions (MCQs) generated by an AI language model and those designed by experienced teachers. Using a quasi-experimental design, two parallel test forms of 30 items each were administered to 300 senior secondary school students in Akwa Ibom State. Parameters analyzed were difficulty index and discrimination index, while test reliability was examined using KR-20, and validity was through content alignment and correlated with external achievement scores. Results revealed that both AI-generated and teacher-made tests achieved acceptable reliability coefficients (α = 0.746 and α = 0.949, respectively). Teacher-made items demonstrated slightly superior discriminating indices (mean = 0.45) compared to AI-generated items (mean = 0.41), whereas AI-generated items exhibited a more balanced difficulty level, with 74% falling within the optimal difficulty range compared to 69% of teacher-made items. The findings indicate that AI-generated MCQs can produce psychometrically sound items comparable to teacher-made ones, though refinement is needed in discriminative power and distractor plausibility. This study concludes that AI holds promise as a supportive tool for large-scale item generation, but human expertise remains essential for ensuring validity and alignment with pedagogical intent.
Zitieren
APA
Evans, G., Evans, U., & Esin, J. O. (2026). Comparative Analysis of Psychometric Profiling of AI-Generated and Teacher Constructed Multiple-Choice Question Items. Journal of Innovation in Educational Assessment, 8(1), 52-70. https://doi.org/10.66545/s7vd5d22
Vancouver
Evans G, Evans U, Esin JO. Comparative Analysis of Psychometric Profiling of AI-Generated and Teacher Constructed Multiple-Choice Question Items. Journal of Innovation in Educational Assessment. 2026;8(1):52-70. doi:10.66545/s7vd5d22
BibTeX
@article{Evans2026,
author = {Evans, Glory and Evans, Udoh and Esin, John Okon},
title = {Comparative Analysis of Psychometric Profiling of AI-Generated and Teacher Constructed Multiple-Choice Question Items},
journal = {Journal of Innovation in Educational Assessment},
year = {2026},
volume = {8},
number = {1},
pages = {52--70},
doi = {10.66545/s7vd5d22}
}
EUVIMED