EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

An Empirical Analysis of Large Language Models for Automated Cross-Prompt Essay Trait Scoring in Brazilian Portuguese

André Barbosa, Igor Cataneo Silveira, Denis Deratani Mauá

Journal of the Brazilian Computer Society · 06.10.2025 · 31 (1) · 858-871

DOI: 10.5753/jbcs.2025.5817 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the Brazilian Computer Society, online erschienen am 06.10.2025. Verfasst von André Barbosa und 2 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The development of automated essay grading systems with minimal human intervention has been pursued for decades. While these systems have advanced significantly in English, there is still a lack of in-depth analysis of the use of modern Large Language Models for automatic essay scoring in Portuguese. This work addresses this gap by evaluating different language model architectures (encoder-only, decoder-only, reasoning-based), fine-tuning and prompt engineering strategies. Our study focuses on scoring argumentative essays written as practice exercises for the Brazilian national entrance exam regarding five trait-specific criteria. Our results show that no architecture is always dominant, and that encoder-only models offer a good balance between accuracy and computational cost. We obtain state-of-the-art results for the dataset, obtaining trait-specific performance that ranges from .60 to .73 measured in Quadratic Weighted Kappa.

Zitieren

APA

Barbosa, A., Silveira, I. C., & Mauá, D. D. (2025). An Empirical Analysis of Large Language Models for Automated Cross-Prompt Essay Trait Scoring in Brazilian Portuguese. Journal of the Brazilian Computer Society, 31(1), 858-871. https://doi.org/10.5753/jbcs.2025.5817

Vancouver

Barbosa A, Silveira IC, Mauá DD. An Empirical Analysis of Large Language Models for Automated Cross-Prompt Essay Trait Scoring in Brazilian Portuguese. Journal of the Brazilian Computer Society. 2025;31(1):858-871. doi:10.5753/jbcs.2025.5817

BibTeX
@article{Barbosa2025,
  author = {Barbosa, André and Silveira, Igor Cataneo and Mauá, Denis Deratani},
  title = {An Empirical Analysis of Large Language Models for Automated Cross-Prompt Essay Trait Scoring in Brazilian Portuguese},
  journal = {Journal of the Brazilian Computer Society},
  year = {2025},
  volume = {31},
  number = {1},
  pages = {858--871},
  doi = {10.5753/jbcs.2025.5817}
}
Kostenlos starten →