Beyond reliability: examining the applicability of adaptive comparative judgment in high-stakes assessment
Frontiers in Education · 13.03.2026 · 11
DOI: 10.3389/feduc.2026.1755046 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Education, online erschienen am 13.03.2026. Verfasst von Kjetil Egelandsdal und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
Adaptive Comparative Judgment (ACJ) has been promoted as a promising alternative to criteria-based grading, yet evidence of its performance in high-stakes, text-intensive higher-education contexts remains limited. This study examines the applicability of ACJ as a grading method for complex, text-based assessments in higher education.
Method
ACJ was applied post hoc to examination papers from two third-year law courses at a Norwegian university ( N = 107) and compared with conventional grades using agreement statistics, discrepant-case expert review, examiner survey data, and judgment-time measures.
Results
ACJ produced high reliability (Scale Separation Reliability = 0.85–0.89) and substantial grade convergence (> 90% identical or adjacent grades), although fewer than half of grades matched exactly and course-specific shifts in grade outcomes were sensitive to boundary placement. Discrepant-case reviews suggested that surface-level features (structure and readability) were often favored over deeper analytical reasoning, consistent with examiner reflections and short median judgment times.
Discussion
Translating rank orders into grades required substantial post-hoc thresholding and coordination work, and the absence of individualized justifications constrained transparency. Overall, while ACJ can yield reliable rankings, its practical use in high-stakes settings involving long written responses raises important validity and feasibility concerns.
Zitieren
APA
Egelandsdal, K., & Færstad, J. (2026). Beyond reliability: examining the applicability of adaptive comparative judgment in high-stakes assessment. Frontiers in Education, 11. https://doi.org/10.3389/feduc.2026.1755046
Vancouver
Egelandsdal K, Færstad J. Beyond reliability: examining the applicability of adaptive comparative judgment in high-stakes assessment. Frontiers in Education. 2026;11. doi:10.3389/feduc.2026.1755046
BibTeX
@article{Egelandsdal2026,
author = {Egelandsdal, Kjetil and Færstad, Jan-Ove},
title = {Beyond reliability: examining the applicability of adaptive comparative judgment in high-stakes assessment},
journal = {Frontiers in Education},
year = {2026},
volume = {11},
doi = {10.3389/feduc.2026.1755046}
}
EUVIMED