condagree: An R Package for Ability-Conditioned Inter-Rater Agreement in Constructed-Response Scoring
Applied Psychological Measurement · 02.11.2026
DOI: 10.1177/01466216261506289 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Psychological Measurement, online erschienen am 02.11.2026. Verfasst von Azmi Fitrisia und 2 weiteren Autor:innen. Beteiligte Einrichtung: History Education, Universitas Negeri Padang, Padang, Indonesia. Die Arbeit nennt 7 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Inter-rater agreement for constructed-response items is usually reported as one index per item, which cannot show whether scoring is less consistent for some examinees than for others. condagree is an R package that estimates agreement as a function of examinee ability and uses a latent-category rater error model to separate a score-opportunity effect from a residual ability effect. In simulations, conditioning on the operational score rejected a true null hypothesis in up to 68% of replications, whereas the latent-model test rejected in at most 8.5%. In 14,708 double-scored PIRLS 2011 responses from Indonesia, kappa fell from .97 to .88 across achievement quintiles, a gradient explained mainly by score opportunity.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- History Education, Universitas Negeri Padang, Padang, Indonesia.
- Department of Educational Research and Evaluation, Graduate School, Universitas Negeri Yogyakarta, Yogyakarta, Indonesia.
- Faculty of Social Sciences and Political Sciences, Universitas Negeri Yogyakarta, Yogyakarta, Indonesia.
ORCID-Profile: Tri Zahra Ningsih
Zitieren
APA
Fitrisia, A., Ningsih, T. Z., & Putri, S. K. (2026). condagree: An R Package for Ability-Conditioned Inter-Rater Agreement in Constructed-Response Scoring. Applied Psychological Measurement. https://doi.org/10.1177/01466216261506289
Vancouver
Fitrisia A, Ningsih TZ, Putri SK. condagree: An R Package for Ability-Conditioned Inter-Rater Agreement in Constructed-Response Scoring. Applied Psychological Measurement. 2026. doi:10.1177/01466216261506289
BibTeX
@article{Fitrisia2026,
author = {Fitrisia, Azmi and Ningsih, Tri Zahra and Putri, Suci Kurnia},
title = {condagree: An R Package for Ability-Conditioned Inter-Rater Agreement in Constructed-Response Scoring},
journal = {Applied Psychological Measurement},
year = {2026},
doi = {10.1177/01466216261506289}
}
EUVIMED