EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Automated Essay Scoring Using Transformer Models

Sabrina Ludwig, Christian Mayer, Christopher Hansen, Kerstin Eilers, Steffen Brandt

Psych · 14.12.2021 · 3 (4) · 897-915

DOI: 10.3390/psych3040056 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Psych, online erschienen am 14.12.2021. Verfasst von Sabrina Ludwig und 4 weiteren Autor:innen. Die Arbeit wurde bisher 64-mal zitiert (Crossref) und nennt 81 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: German Federal Ministry of Education and Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automated essay scoring (AES) is gaining increasing attention in the education sector as it significantly reduces the burden of manual scoring and allows ad hoc feedback for learners. Natural language processing based on machine learning has been shown to be particularly suitable for text classification and AES. While many machine-learning approaches for AES still rely on a bag of words (BOW) approach, we consider a transformer-based approach in this paper, compare its performance to a logistic regression model based on the BOW approach, and discuss their differences. The analysis is based on 2088 email responses to a problem-solving task that were manually labeled in terms of politeness. Both transformer models considered in the analysis outperformed without any hyperparameter tuning of the regression-based model. We argue that, for AES tasks such as politeness classification, the transformer-based approach has significant advantages, while a BOW approach suffers from not taking word order into account and reducing the words to their stem. Further, we show how such models can help increase the accuracy of human raters, and we provide a detailed instruction on how to implement transformer-based models for one’s own purposes.

Förderung

  • German Federal Ministry of Education and Research

Zitieren

APA

Ludwig, S., Mayer, C., Hansen, C., Eilers, K., & Brandt, S. (2021). Automated Essay Scoring Using Transformer Models. Psych, 3(4), 897-915. https://doi.org/10.3390/psych3040056

Vancouver

Ludwig S, Mayer C, Hansen C, Eilers K, Brandt S. Automated Essay Scoring Using Transformer Models. Psych. 2021;3(4):897-915. doi:10.3390/psych3040056

BibTeX
@article{Ludwig2021,
  author = {Ludwig, Sabrina and Mayer, Christian and Hansen, Christopher and Eilers, Kerstin and Brandt, Steffen},
  title = {Automated Essay Scoring Using Transformer Models},
  journal = {Psych},
  year = {2021},
  volume = {3},
  number = {4},
  pages = {897--915},
  doi = {10.3390/psych3040056}
}
Kostenlos starten →