EUVIMEDEUVIMED entdecken
Open Access

condagree: An R Package for Ability-Conditioned Inter-Rater Agreement in Constructed-Response Scoring

Azmi Fitrisia, Tri Zahra Ningsih iD, Suci Kurnia Putri

Applied Psychological Measurement · 02.11.2026

DOI: 10.1177/01466216261506289 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Psychological Measurement, online erschienen am 02.11.2026. Verfasst von Azmi Fitrisia und 2 weiteren Autor:innen. Beteiligte Einrichtung: History Education, Universitas Negeri Padang, Padang, Indonesia. Die Arbeit nennt 7 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Inter-rater agreement for constructed-response items is usually reported as one index per item, which cannot show whether scoring is less consistent for some examinees than for others. condagree is an R package that estimates agreement as a function of examinee ability and uses a latent-category rater error model to separate a score-opportunity effect from a residual ability effect. In simulations, conditioning on the operational score rejected a true null hypothesis in up to 68% of replications, whereas the latent-model test rejected in at most 8.5%. In 14,708 double-scored PIRLS 2011 responses from Indonesia, kappa fell from .97 to .88 across achievement quintiles, a gradient explained mainly by score opportunity.

Schlagwörter

Stichwörter der Autor:innen

R packageconstructed responseinter-rater agreementplausible valuesrater effectsscoring reliability

Autor:innen und Institutionen

  • History Education, Universitas Negeri Padang, Padang, Indonesia.
  • Department of Educational Research and Evaluation, Graduate School, Universitas Negeri Yogyakarta, Yogyakarta, Indonesia.
  • Faculty of Social Sciences and Political Sciences, Universitas Negeri Yogyakarta, Yogyakarta, Indonesia.

ORCID-Profile: Tri Zahra Ningsih

Zitieren

APA

Fitrisia, A., Ningsih, T. Z., & Putri, S. K. (2026). condagree: An R Package for Ability-Conditioned Inter-Rater Agreement in Constructed-Response Scoring. Applied Psychological Measurement. https://doi.org/10.1177/01466216261506289

Vancouver

Fitrisia A, Ningsih TZ, Putri SK. condagree: An R Package for Ability-Conditioned Inter-Rater Agreement in Constructed-Response Scoring. Applied Psychological Measurement. 2026. doi:10.1177/01466216261506289

BibTeX
@article{Fitrisia2026,
  author = {Fitrisia, Azmi and Ningsih, Tri Zahra and Putri, Suci Kurnia},
  title = {condagree: An R Package for Ability-Conditioned Inter-Rater Agreement in Constructed-Response Scoring},
  journal = {Applied Psychological Measurement},
  year = {2026},
  doi = {10.1177/01466216261506289}
}
Kostenlos starten →