EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Indonesian Automated Essay Scoring: A Comparative Study of Pretrained Transformer Models

Pulung Hendro Prastyo, Eddy Tungadi, Shaifudin Zuhdi

Information Technology Education Journal · 02.06.2025 · 120-130

DOI: 10.59562/intec.v4i2.8069 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information Technology Education Journal, online erschienen am 02.06.2025. Verfasst von Pulung Hendro Prastyo und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Manual essay scoring is often characterized by inefficiency and inconsistency. This process is notably time-consuming, leading to delayed feedback and increased susceptibility to evaluator fatigue and subjective bias, thereby posing significant challenges. Automated Essay Scoring (AES) offers a scalable, robust, and consistent solution to these issues. However, the performance of AES models can vary considerably depending on the specific application. Therefore, this study evaluated ten Indonesian pretrained transformer models from Hugging Face for AES tasks, using 300 essay responses from a Research Methodology quiz at Politeknik Negeri Ujung Pandang. Performance was assessed using Root Mean Square Error (RMSE) and Quadratic Weighted Kappa (QWK). Among the evaluated models, Indobenchmark/indobert-base-p2 (BERT-02) demonstrated superior performance. It achieved the lowest RMSE of 5.664 and the highest QWK score of 0.6745. The findings suggest that BERT-02 is the most effective model for Indonesian AES tasks. Future research could explore larger datasets and different models to further enhance the performance and understanding of Indonesian AES systems.

Zitieren

APA

Prastyo, P. H., Tungadi, E., & Zuhdi, S. (2025). Indonesian Automated Essay Scoring: A Comparative Study of Pretrained Transformer Models. Information Technology Education Journal, 120-130. https://doi.org/10.59562/intec.v4i2.8069

Vancouver

Prastyo PH, Tungadi E, Zuhdi S. Indonesian Automated Essay Scoring: A Comparative Study of Pretrained Transformer Models. Information Technology Education Journal. 2025:120-130. doi:10.59562/intec.v4i2.8069

BibTeX
@article{Prastyo2025,
  author = {Prastyo, Pulung Hendro and Tungadi, Eddy and Zuhdi, Shaifudin},
  title = {Indonesian Automated Essay Scoring: A Comparative Study of Pretrained Transformer Models},
  journal = {Information Technology Education Journal},
  year = {2025},
  pages = {120--130},
  doi = {10.59562/intec.v4i2.8069}
}
Kostenlos starten →