Predicting Operational Rater‐Type Classifications Using Rasch Measurement Theory and Random Forests: A Music Performance Assessment Perspective
Journal of Educational Measurement · 2019-09 · 56 (3) · 610-625
DOI: 10.1111/jedm.12227 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Educational Measurement, online erschienen am 05.08.2019. Verfasst von Brian C. Wesolowski. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 44 Literaturangaben.
Abstract (Originalsprache)
The purpose of this study was to build a Random Forest supervised machine learning model in order to predict musical rater‐type classifications based upon a Rasch analysis of raters’ differential severity/leniency related to item use. Raw scores (N = 1,704) from 142 raters across nine high school solo and ensemble festivals (grades 9–12) were collected using a 29‐item Likert‐type rating scale embedded within five domains (tone/intonation, n = 6; balance, n = 5; interpretation, n = 6; rhythm, n = 6; and technical accuracy, n = 6). Data were analyzed using a Many Facets Rasch Partial Credit Model. An a priori k‐means cluster analysis of 29 differential rater functioning indices produced a discrete feature vector that classified raters into one of three distinct rater‐types: (a) syntactical rater‐type, (b) expressive rater‐type, or (c) mental representation rater‐type. Results of the initial Random Forest model resulted in an out‐of‐bag error rate of 5.05%, indicating that approximately 95% of the raters were correctly classified. After tuning a set of three hyperparameters (n tree , m try , and node size), the optimized model demonstrated an improved out‐of‐bag error rate of 2.02%. Implications for improvements in assessment, research, and rater training in the field of music education are discussed.
Zitieren
APA
Wesolowski, B. C. (2019). Predicting Operational Rater‐Type Classifications Using Rasch Measurement Theory and Random Forests: A Music Performance Assessment Perspective. Journal of Educational Measurement, 56(3), 610-625. https://doi.org/10.1111/jedm.12227
Vancouver
Wesolowski BC. Predicting Operational Rater‐Type Classifications Using Rasch Measurement Theory and Random Forests: A Music Performance Assessment Perspective. Journal of Educational Measurement. 2019;56(3):610-625. doi:10.1111/jedm.12227
BibTeX
@article{Wesolowski2019,
author = {Wesolowski, Brian C.},
title = {Predicting Operational Rater‐Type Classifications Using Rasch Measurement Theory and Random Forests: A Music Performance Assessment Perspective},
journal = {Journal of Educational Measurement},
year = {2019},
volume = {56},
number = {3},
pages = {610--625},
doi = {10.1111/jedm.12227}
}
EUVIMED