EUVIMEDEUVIMED entdecken
Crossref

Understanding Mean Score Differences Between the e‐rater ® Automated Scoring Engine and Humans for Demographically Based Groups in the GRE ® General Test

Chaitanya Ramineni, David Williamson

ETS Research Report Series · 2018-12 · 2018 (1) · 1-31

DOI: 10.1002/ets2.12192 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ETS Research Report Series, online erschienen am 27.04.2018. Verfasst von Chaitanya Ramineni und 1 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 40 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Notable mean score differences for the e‐rater ® automated scoring engine and for humans for essays from certain demographic groups were observed for the GRE ® General Test in use before the major revision of 2012, called rGRE. The use of e‐rater as a check‐score model with discrepancy thresholds prevented an adverse impact on the examinee score at the item or test level. Despite this control, there remains a need to understand the root causes of these demographically based score differences and to identify potential mechanisms for avoiding future instances of discrepancy. In this study, we used a combination of statistical methods and human review to propose hypotheses about the root cause of score differences and whether such discrepancies reflect inadequacies of e‐rater, human scoring, or both. The human rating process was found to be influenced strongly by the scale structure and did not fully correspond to the e‐rater scoring mechanism. The human raters appeared to be using conditional logic and a rule‐based approach to their scoring, while e‐rater uses linear weighting of all the features. These analyses have implications for future research and operational policies for the scoring of the rGRE.

Zitieren

APA

Ramineni, C., & Williamson, D. (2018). Understanding Mean Score Differences Between the e‐rater ® Automated Scoring Engine and Humans for Demographically Based Groups in the GRE ® General Test. ETS Research Report Series, 2018(1), 1-31. https://doi.org/10.1002/ets2.12192

Vancouver

Ramineni C, Williamson D. Understanding Mean Score Differences Between the e‐rater ® Automated Scoring Engine and Humans for Demographically Based Groups in the GRE ® General Test. ETS Research Report Series. 2018;2018(1):1-31. doi:10.1002/ets2.12192

BibTeX
@article{Ramineni2018,
  author = {Ramineni, Chaitanya and Williamson, David},
  title = {Understanding Mean Score Differences Between the e‐rater ® Automated Scoring Engine and Humans for Demographically Based Groups in the GRE ® General Test},
  journal = {ETS Research Report Series},
  year = {2018},
  volume = {2018},
  number = {1},
  pages = {1--31},
  doi = {10.1002/ets2.12192}
}
Infokarte | EUVIMED

Artikeldaten derzeit nicht verfügbar.

Recherche