Evaluating LLMs for Automated Scoring in Formative Assessments
Applied Sciences · 05.03.2025 · 15 (5) · 2787
DOI: 10.3390/app15052787 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 05.03.2025. Verfasst von Pedro C. Mendonça und 2 weiteren Autor:innen. Die Arbeit wurde bisher 56-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundação para a Ciência e a Tecnologia (FCT).
Abstract (Originalsprache)
The increasing complexity and scale of modern education have revealed the shortcomings of traditional grading methods in providing consistent and scalable assessments. Advancements in artificial intelligence have positioned Large Language Models (LLMs) as robust solutions for automating grading tasks. This study systematically compared the grading performance of an open-source LLM (LLaMA 3.2) and a premium LLM (OpenAI GPT-4o) against human evaluators across diverse question types in the context of a computer programming subject. Using detailed rubrics, the study assessed the alignment between LLM-generated and human-assigned grades. Results revealed that while both LLMs align closely with human grading, equivalence testing demonstrated that the premium LLM achieves statistically and practically similar grading patterns, particularly for code-based questions, suggesting its potential as a reliable tool for educational assessments. These findings underscore the ability of LLMs to enhance grading consistency, reduce educator workload, and address scalability challenges in programming-focused assessments.
Förderung
- Fundação para a Ciência e a Tecnologia (FCT)
Zitieren
APA
Mendonça, P. C., Quintal, F., & Mendonça, F. (2025). Evaluating LLMs for Automated Scoring in Formative Assessments. Applied Sciences, 15(5), 2787. https://doi.org/10.3390/app15052787
Vancouver
Mendonça PC, Quintal F, Mendonça F. Evaluating LLMs for Automated Scoring in Formative Assessments. Applied Sciences. 2025;15(5):2787. doi:10.3390/app15052787
BibTeX
@article{Mendona2025,
author = {Mendonça, Pedro C. and Quintal, Filipe and Mendonça, Fábio},
title = {Evaluating LLMs for Automated Scoring in Formative Assessments},
journal = {Applied Sciences},
year = {2025},
volume = {15},
number = {5},
pages = {2787},
doi = {10.3390/app15052787}
}
EUVIMED