An Empirical Analysis of Large Language Models for Automated Cross-Prompt Essay Trait Scoring in Brazilian Portuguese
Journal of the Brazilian Computer Society · 06.10.2025 · 31 (1) · 858-871
DOI: 10.5753/jbcs.2025.5817 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the Brazilian Computer Society, online erschienen am 06.10.2025. Verfasst von André Barbosa und 2 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The development of automated essay grading systems with minimal human intervention has been pursued for decades. While these systems have advanced significantly in English, there is still a lack of in-depth analysis of the use of modern Large Language Models for automatic essay scoring in Portuguese. This work addresses this gap by evaluating different language model architectures (encoder-only, decoder-only, reasoning-based), fine-tuning and prompt engineering strategies. Our study focuses on scoring argumentative essays written as practice exercises for the Brazilian national entrance exam regarding five trait-specific criteria. Our results show that no architecture is always dominant, and that encoder-only models offer a good balance between accuracy and computational cost. We obtain state-of-the-art results for the dataset, obtaining trait-specific performance that ranges from .60 to .73 measured in Quadratic Weighted Kappa.
Zitieren
APA
Barbosa, A., Silveira, I. C., & Mauá, D. D. (2025). An Empirical Analysis of Large Language Models for Automated Cross-Prompt Essay Trait Scoring in Brazilian Portuguese. Journal of the Brazilian Computer Society, 31(1), 858-871. https://doi.org/10.5753/jbcs.2025.5817
Vancouver
Barbosa A, Silveira IC, Mauá DD. An Empirical Analysis of Large Language Models for Automated Cross-Prompt Essay Trait Scoring in Brazilian Portuguese. Journal of the Brazilian Computer Society. 2025;31(1):858-871. doi:10.5753/jbcs.2025.5817
BibTeX
@article{Barbosa2025,
author = {Barbosa, André and Silveira, Igor Cataneo and Mauá, Denis Deratani},
title = {An Empirical Analysis of Large Language Models for Automated Cross-Prompt Essay Trait Scoring in Brazilian Portuguese},
journal = {Journal of the Brazilian Computer Society},
year = {2025},
volume = {31},
number = {1},
pages = {858--871},
doi = {10.5753/jbcs.2025.5817}
}
EUVIMED