EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating LLMs for Automated Scoring in Formative Assessments

Pedro C. Mendonça, Filipe Quintal, Fábio Mendonça

Applied Sciences · 05.03.2025 · 15 (5) · 2787

DOI: 10.3390/app15052787 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 05.03.2025. Verfasst von Pedro C. Mendonça und 2 weiteren Autor:innen. Die Arbeit wurde bisher 56-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundação para a Ciência e a Tecnologia (FCT).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The increasing complexity and scale of modern education have revealed the shortcomings of traditional grading methods in providing consistent and scalable assessments. Advancements in artificial intelligence have positioned Large Language Models (LLMs) as robust solutions for automating grading tasks. This study systematically compared the grading performance of an open-source LLM (LLaMA 3.2) and a premium LLM (OpenAI GPT-4o) against human evaluators across diverse question types in the context of a computer programming subject. Using detailed rubrics, the study assessed the alignment between LLM-generated and human-assigned grades. Results revealed that while both LLMs align closely with human grading, equivalence testing demonstrated that the premium LLM achieves statistically and practically similar grading patterns, particularly for code-based questions, suggesting its potential as a reliable tool for educational assessments. These findings underscore the ability of LLMs to enhance grading consistency, reduce educator workload, and address scalability challenges in programming-focused assessments.

Förderung

  • Fundação para a Ciência e a Tecnologia (FCT)

Zitieren

APA

Mendonça, P. C., Quintal, F., & Mendonça, F. (2025). Evaluating LLMs for Automated Scoring in Formative Assessments. Applied Sciences, 15(5), 2787. https://doi.org/10.3390/app15052787

Vancouver

Mendonça PC, Quintal F, Mendonça F. Evaluating LLMs for Automated Scoring in Formative Assessments. Applied Sciences. 2025;15(5):2787. doi:10.3390/app15052787

BibTeX
@article{Mendona2025,
  author = {Mendonça, Pedro C. and Quintal, Filipe and Mendonça, Fábio},
  title = {Evaluating LLMs for Automated Scoring in Formative Assessments},
  journal = {Applied Sciences},
  year = {2025},
  volume = {15},
  number = {5},
  pages = {2787},
  doi = {10.3390/app15052787}
}
Kostenlos starten →