Predicting Item Difficulty and Learner Performance on a Bilingual Gamified Quiz Platform
International Journal of Advanced Computer Science and Applications · 2026 · 17 (10)
DOI: 10.14569/ijacsa.2026.0171014 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Advanced Computer Science and Applications, 2026. Verfasst von Aibolat Kulatay und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Gamified quiz platforms need estimates of question difficulty before response data exist, and learner ratings that support personalized practice. Three related analyses were carried out on separate data sets from a Kazakh–Russian quiz platform. First, lexical and transformer models predicted the observed difficulty of 30,444 items, derived from 177.5 million recorded attempts. Second, item response theory (IRT) models were fitted to 67,990 tournament responses, and text-based predictions were compared with IRT difficulty for 327 new items. Third, online rating models were evaluated on 100,000 duels and 322,046 reconstructed question responses. A fine-tuned XLM-RoBERTa-large ensemble predicted observed difficulty with a Pearson correlation of 0.527, against 0.461 for a lexical baseline (difference 0.066; 95% confidence interval 0.032–0.099). For new tournament items, correlations with two-parameter logistic (2PL) IRT difficulty were 0.24–0.33; the predictions identified the hardest tenth of items better than chance (area under the ROC curve, AUC, 0.66–0.69), but in a cross-fitted calibration simulation they improved on a population prior only before any responses were observed. Observed difficulty correlated at 0.94 between 4,963 Kazakh originals and their Russian translations, which were harder on average. Information criteria did not separate the one- and two-parameter IRT models. Elo predicted duel outcomes with an AUC of 0.828, slightly above tuned TrueSkill and Glicko-2 in the configurations tested, and a learner-by-category term raised the AUC for response correctness from 0.684 to 0.697. Text-based estimates therefore suit coarse screening of new items, and the ratings predict performance but were not validated as measures of proficiency.
Zitieren
APA
Kulatay, A., Zhaisanova, D., Kulambayev, B., & Mambetov, S. (2026). Predicting Item Difficulty and Learner Performance on a Bilingual Gamified Quiz Platform. International Journal of Advanced Computer Science and Applications, 17(10). https://doi.org/10.14569/ijacsa.2026.0171014
Vancouver
Kulatay A, Zhaisanova D, Kulambayev B, Mambetov S. Predicting Item Difficulty and Learner Performance on a Bilingual Gamified Quiz Platform. International Journal of Advanced Computer Science and Applications. 2026;17(10). doi:10.14569/ijacsa.2026.0171014
BibTeX
@article{Kulatay2026,
author = {Kulatay, Aibolat and Zhaisanova, Dinara and Kulambayev, Bakhytzhan and Mambetov, Saken},
title = {Predicting Item Difficulty and Learner Performance on a Bilingual Gamified Quiz Platform},
journal = {International Journal of Advanced Computer Science and Applications},
year = {2026},
volume = {17},
number = {10},
doi = {10.14569/ijacsa.2026.0171014}
}
EUVIMED