The Effects of Incomplete Rating Designs in Combination With Rater Effects
Journal of Educational Measurement · 2019-03 · 56 (1) · 76-100
DOI: 10.1111/jedm.12201 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Educational Measurement, online erschienen am 13.03.2019. Verfasst von Stefanie A. Wind und 1 weiteren Autor:innen. Die Arbeit wurde bisher 40-mal zitiert (Crossref) und nennt 49 Literaturangaben.
Abstract (Originalsprache)
Researchers have explored a variety of topics related to identifying and distinguishing among specific types of rater effects, as well as the implications of different types of incomplete data collection designs for rater‐mediated assessments. In this study, we used simulated data to examine the sensitivity of latent trait model indicators of three rater effects (leniency, central tendency, and severity) in combination with different types of incomplete rating designs (systematic links, anchor performances, and spiral). We used the rating scale model and the partial credit model to calculate rater location estimates, standard errors of rater estimates, model–data fit statistics, and the standard deviation of rating scale category thresholds as indicators of rater effects and we explored the sensitivity of these indicators to rater effects under different conditions. Our results suggest that it is possible to detect rater effects when each of the three types of rating designs is used. However, there are differences in the sensitivity of each indicator related to type of rater effect, type of rating design, and the overall proportion of effect raters. We discuss implications for research and practice related to rater‐mediated assessments.
Zitieren
APA
Wind, S. A., & Jones, E. (2019). The Effects of Incomplete Rating Designs in Combination With Rater Effects. Journal of Educational Measurement, 56(1), 76-100. https://doi.org/10.1111/jedm.12201
Vancouver
Wind SA, Jones E. The Effects of Incomplete Rating Designs in Combination With Rater Effects. Journal of Educational Measurement. 2019;56(1):76-100. doi:10.1111/jedm.12201
BibTeX
@article{Wind2019,
author = {Wind, Stefanie A. and Jones, Eli},
title = {The Effects of Incomplete Rating Designs in Combination With Rater Effects},
journal = {Journal of Educational Measurement},
year = {2019},
volume = {56},
number = {1},
pages = {76--100},
doi = {10.1111/jedm.12201}
}
EUVIMED