EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Linking essay-writing tests using many-facet models and neural automated essay scoring

Masaki Uto iD, Kota Aramaki

Behavior Research Methods · 2024

DOI: 10.3758/s13428-024-02485-2 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Behavior Research Methods, 2024. Verfasst von Masaki Uto und 1 weiteren Autor:innen. Beteiligte Einrichtung: The University of Electro-Communications, Tokyo, Japan. uto@ai.lab.uec.ac.jp. Hauptthemen laut MEDLINE: Writing. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Society for the Promotion of Science.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

For essay-writing tests, challenges arise when scores assigned to essays are influenced by the characteristics of raters, such as rater severity and consistency. Item response theory (IRT) models incorporating rater parameters have been developed to tackle this issue, exemplified by the many-facet Rasch models. These IRT models enable the estimation of examinees’ abilities while accounting for the impact of rater characteristics, thereby enhancing the accuracy of ability measurement. However, difficulties can arise when different groups of examinees are evaluated by different sets of raters. In such cases, test linking is essential for unifying the scale of model parameters estimated for individual examinee–rater groups. Traditional test-linking methods typically require administrators to design groups in which either examinees or raters are partially shared. However, this is often impractical in real-world testing scenarios. To address this, we introduce a novel method for linking the parameters of IRT models with rater parameters that uses neural automated essay scoring technology. Our experimental results indicate that our method successfully accomplishes test linking with accuracy comparable to that of linear linking using few common examinees.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

WritingHumansEducational MeasurementPsychometricsModels, Statistical

Stichwörter der Autor:innen

Automated essay scoringEducational measurementIRT linkingItem response theoryMany-facet Rasch modelsWriting assessment

Autor:innen und Institutionen

  • The University of Electro-Communications, Tokyo, Japan. uto@ai.lab.uec.ac.jp.
  • The University of Electro-Communications, Tokyo, Japan.

ORCID-Profile: Masaki Uto

Förderung

  • Japan Society for the Promotion of Science

Zitieren

APA

Uto, M., & Aramaki, K. (2024). Linking essay-writing tests using many-facet models and neural automated essay scoring. Behavior Research Methods. https://doi.org/10.3758/s13428-024-02485-2

Vancouver

Uto M, Aramaki K. Linking essay-writing tests using many-facet models and neural automated essay scoring. Behavior Research Methods. 2024. doi:10.3758/s13428-024-02485-2

BibTeX
@article{Uto2024,
  author = {Uto, Masaki and Aramaki, Kota},
  title = {Linking essay-writing tests using many-facet models and neural automated essay scoring},
  journal = {Behavior Research Methods},
  year = {2024},
  doi = {10.3758/s13428-024-02485-2}
}
Kostenlos starten →