EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Using Confidence Modeling to Optimize Overall Score Quality in Hybrid Scoring Systems

Alexander Kwako, Susan Lottridge, Christopher Ormerod

Educational Measurement: Issues and Practice · 2026-06 · 45 (2)

DOI: 10.1111/emip.70019 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Educational Measurement: Issues and Practice, online erschienen am 05.04.2026. Verfasst von Alexander Kwako und 2 weiteren Autor:innen. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In large‐scale assessments, constructed response items are often scored using hybrid scoring systems, which combine human and automated scores. In this study, we augment automated scoring with confidence modeling to strategically route difficult‐to‐score responses for human review. We utilize hybrid performance curves to visualize the impact of routing on performance. Additionally, we propose several hybrid scoring policies for selecting optimal routing thresholds given practical constraints. Our findings reveal that hybrid scoring systems can achieve an overall performance that exceeds that of human‐ and automated‐only systems. Moreover, the superior performance of the hybrid system is less expensive than a human‐only system. These findings highlight the complementarity of human raters and automated scoring engines. Although current standards focus on the performance of human raters and automated scoring engines in isolation, we recommend that practitioners also report on the performance of the hybrid scoring system as a whole.

Zitieren

APA

Kwako, A., Lottridge, S., & Ormerod, C. (2026). Using Confidence Modeling to Optimize Overall Score Quality in Hybrid Scoring Systems. Educational Measurement: Issues and Practice, 45(2). https://doi.org/10.1111/emip.70019

Vancouver

Kwako A, Lottridge S, Ormerod C. Using Confidence Modeling to Optimize Overall Score Quality in Hybrid Scoring Systems. Educational Measurement: Issues and Practice. 2026;45(2). doi:10.1111/emip.70019

BibTeX
@article{Kwako2026,
  author = {Kwako, Alexander and Lottridge, Susan and Ormerod, Christopher},
  title = {Using Confidence Modeling to Optimize Overall Score Quality in Hybrid Scoring Systems},
  journal = {Educational Measurement: Issues and Practice},
  year = {2026},
  volume = {45},
  number = {2},
  doi = {10.1111/emip.70019}
}
Kostenlos starten →