Linking essay-writing tests using many-facet models and neural automated essay scoring
Behavior Research Methods · 2024
DOI: 10.3758/s13428-024-02485-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift Behavior Research Methods, 2024. Verfasst von Masaki Uto und 1 weiteren Autor:innen. Beteiligte Einrichtung: The University of Electro-Communications, Tokyo, Japan. uto@ai.lab.uec.ac.jp. Hauptthemen laut MEDLINE: Writing. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Society for the Promotion of Science.
Abstract (Originalsprache)
For essay-writing tests, challenges arise when scores assigned to essays are influenced by the characteristics of raters, such as rater severity and consistency. Item response theory (IRT) models incorporating rater parameters have been developed to tackle this issue, exemplified by the many-facet Rasch models. These IRT models enable the estimation of examinees’ abilities while accounting for the impact of rater characteristics, thereby enhancing the accuracy of ability measurement. However, difficulties can arise when different groups of examinees are evaluated by different sets of raters. In such cases, test linking is essential for unifying the scale of model parameters estimated for individual examinee–rater groups. Traditional test-linking methods typically require administrators to design groups in which either examinees or raters are partially shared. However, this is often impractical in real-world testing scenarios. To address this, we introduce a novel method for linking the parameters of IRT models with rater parameters that uses neural automated essay scoring technology. Our experimental results indicate that our method successfully accomplishes test linking with accuracy comparable to that of linear linking using few common examinees.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- The University of Electro-Communications, Tokyo, Japan. uto@ai.lab.uec.ac.jp.
- The University of Electro-Communications, Tokyo, Japan.
ORCID-Profile: Masaki Uto
Förderung
- Japan Society for the Promotion of Science
Zitieren
APA
Uto, M., & Aramaki, K. (2024). Linking essay-writing tests using many-facet models and neural automated essay scoring. Behavior Research Methods. https://doi.org/10.3758/s13428-024-02485-2
Vancouver
Uto M, Aramaki K. Linking essay-writing tests using many-facet models and neural automated essay scoring. Behavior Research Methods. 2024. doi:10.3758/s13428-024-02485-2
BibTeX
@article{Uto2024,
author = {Uto, Masaki and Aramaki, Kota},
title = {Linking essay-writing tests using many-facet models and neural automated essay scoring},
journal = {Behavior Research Methods},
year = {2024},
doi = {10.3758/s13428-024-02485-2}
}
EUVIMED