Development of a large language model-based diagnostic assessment system for detecting students' initial abilities
Measurement In Educational Research · 07.05.2026 · 6 (1)
DOI: 10.33292/meter.v6i1.540 ↗Kurz gefasst
Fachartikel in der Zeitschrift Measurement In Educational Research, online erschienen am 07.05.2026. Verfasst von Anak Agung Gede Pradnya Prameswara und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
The research aimed to develop and evaluate DIGIDAWS (Digital Diagnostic Assessment Web System), a web - based system integrating Google Gemini Large Language Models (LLM) with the Multistage Test (MST) framework through a Retrieval - Augmented Generation (RAG) mechanism to map students’ initial abilities in Informatics education. The method used in this study is the ADDIE (Analysis, Design, Development, Implementation, and Evaluation) Research and Development model. Data evaluation was carried out by involving 108 Grade X students and two Informatics teachers at SMAN 1 Blahbatuh, alongside four academic expert validators using the Gregory formula, black box functional scenarios, and the User Experience Questionnaire Short (UEQ- S). The results showed that expert validation yielded a very high Gregory validity coefficient (Vc = 1.00). In addition, black box testing achieved a 100% success rate across 20 scenarios, and the UEQ- S produced Excellent mean scores for both pragmatic (2.26) and hedonic (2.27) qualities. T he conclusion of this study is that DIGIDAWS was successfully developed as a valid, functional, and well -received diagnostic assessment system. This suggests that integrating LLM - based item generation with the MST framework offers a feasible approach to re ducing teacher administrative burden while supporting initial - ability mapping .
Zitieren
APA
Prameswara, A. A. G. P., Wiradika, I. N. I., Damayanthi, L. P. E., & Pastika, I. P. G. P. (2026). Development of a large language model-based diagnostic assessment system for detecting students' initial abilities. Measurement In Educational Research, 6(1). https://doi.org/10.33292/meter.v6i1.540
Vancouver
Prameswara AAGP, Wiradika INI, Damayanthi LPE, Pastika IPGP. Development of a large language model-based diagnostic assessment system for detecting students' initial abilities. Measurement In Educational Research. 2026;6(1). doi:10.33292/meter.v6i1.540
BibTeX
@article{Prameswara2026,
author = {Prameswara, Anak Agung Gede Pradnya and Wiradika, I Nyoman Indhi and Damayanthi, Luh Putu Eka and Pastika, I Putu Gede Pasek},
title = {Development of a large language model-based diagnostic assessment system for detecting students' initial abilities},
journal = {Measurement In Educational Research},
year = {2026},
volume = {6},
number = {1},
doi = {10.33292/meter.v6i1.540}
}
EUVIMED