EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Development of a large language model-based diagnostic assessment system for detecting students' initial abilities

Anak Agung Gede Pradnya Prameswara, I Nyoman Indhi Wiradika, Luh Putu Eka Damayanthi, I Putu Gede Pasek Pastika

Measurement In Educational Research · 07.05.2026 · 6 (1)

DOI: 10.33292/meter.v6i1.540 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Measurement In Educational Research, online erschienen am 07.05.2026. Verfasst von Anak Agung Gede Pradnya Prameswara und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The research aimed to develop and evaluate DIGIDAWS (Digital Diagnostic Assessment Web System), a web - based system integrating Google Gemini Large Language Models (LLM) with the Multistage Test (MST) framework through a Retrieval - Augmented Generation (RAG) mechanism to map students’ initial abilities in Informatics education. The method used in this study is the ADDIE (Analysis, Design, Development, Implementation, and Evaluation) Research and Development model. Data evaluation was carried out by involving 108 Grade X students and two Informatics teachers at SMAN 1 Blahbatuh, alongside four academic expert validators using the Gregory formula, black box functional scenarios, and the User Experience Questionnaire Short (UEQ- S). The results showed that expert validation yielded a very high Gregory validity coefficient (Vc = 1.00). In addition, black box testing achieved a 100% success rate across 20 scenarios, and the UEQ- S produced Excellent mean scores for both pragmatic (2.26) and hedonic (2.27) qualities. T he conclusion of this study is that DIGIDAWS was successfully developed as a valid, functional, and well -received diagnostic assessment system. This suggests that integrating LLM - based item generation with the MST framework offers a feasible approach to re ducing teacher administrative burden while supporting initial - ability mapping .

Zitieren

APA

Prameswara, A. A. G. P., Wiradika, I. N. I., Damayanthi, L. P. E., & Pastika, I. P. G. P. (2026). Development of a large language model-based diagnostic assessment system for detecting students' initial abilities. Measurement In Educational Research, 6(1). https://doi.org/10.33292/meter.v6i1.540

Vancouver

Prameswara AAGP, Wiradika INI, Damayanthi LPE, Pastika IPGP. Development of a large language model-based diagnostic assessment system for detecting students' initial abilities. Measurement In Educational Research. 2026;6(1). doi:10.33292/meter.v6i1.540

BibTeX
@article{Prameswara2026,
  author = {Prameswara, Anak Agung Gede Pradnya and Wiradika, I Nyoman Indhi and Damayanthi, Luh Putu Eka and Pastika, I Putu Gede Pasek},
  title = {Development of a large language model-based diagnostic assessment system for detecting students' initial abilities},
  journal = {Measurement In Educational Research},
  year = {2026},
  volume = {6},
  number = {1},
  doi = {10.33292/meter.v6i1.540}
}
Kostenlos starten →