Comparative analysis of language models in automated educational diagnostic tasks
Applied Journal of Digital Innovation · 14.03.2026 · 2 · 13
DOI: 10.65835/aj.2026.2.13 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Journal of Digital Innovation, online erschienen am 14.03.2026. Verfasst von María Teresa Nagua Velepucha. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This study presents a comparative analysis of language models for automated educational diagnosis, evaluating two specialized models (a feature-based ML model and a fine-tuned BERT) against three state-of-the-art Large Language Models (GPT-4, Claude 3, Gemini Pro). Using a mixed-methods approach on a corpus of 450 annotated student texts, the research assessed diagnostic accuracy, feedback quality, robustness, and hallucination rates. Results indicate that advanced LLMs, particularly GPT-4 and Claude 3 Opus, achieve superior diagnostic performance (F1-Score ≥ 0.83) and generate pedagogically higher-quality feedback, rated as more useful and actionable by expert judges. However, these models exhibit a critical weakness: a significant propensity for factual hallucinations (5.2% - 12.3%). In contrast, specialized models, while less sophisticated, offer near-perfect reliability and greater cost-efficiency. The study concludes that LLMs represent a qualitative leap in automated assessment but are not yet reliable for autonomous application. The most viable implementation model is AI-augmented education, where LLMs act as diagnostic assistants to teaching professionals, who provide essential oversight, validation, and personalization. This human-in-the-loop paradigm balances technological potential with pedagogical integrity and ethical responsibility.
Zitieren
APA
Velepucha, M. T. N. (2026). Comparative analysis of language models in automated educational diagnostic tasks. Applied Journal of Digital Innovation, 2, 13. https://doi.org/10.65835/aj.2026.2.13
Vancouver
Velepucha MTN. Comparative analysis of language models in automated educational diagnostic tasks. Applied Journal of Digital Innovation. 2026;2:13. doi:10.65835/aj.2026.2.13
BibTeX
@article{Velepucha2026,
author = {Velepucha, María Teresa Nagua},
title = {Comparative analysis of language models in automated educational diagnostic tasks},
journal = {Applied Journal of Digital Innovation},
year = {2026},
volume = {2},
pages = {13},
doi = {10.65835/aj.2026.2.13}
}
EUVIMED