EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Beyond reliability: examining the applicability of adaptive comparative judgment in high-stakes assessment

Kjetil Egelandsdal, Jan-Ove Færstad

Frontiers in Education · 13.03.2026 · 11

DOI: 10.3389/feduc.2026.1755046 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Education, online erschienen am 13.03.2026. Verfasst von Kjetil Egelandsdal und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Adaptive Comparative Judgment (ACJ) has been promoted as a promising alternative to criteria-based grading, yet evidence of its performance in high-stakes, text-intensive higher-education contexts remains limited. This study examines the applicability of ACJ as a grading method for complex, text-based assessments in higher education.

Method

ACJ was applied post hoc to examination papers from two third-year law courses at a Norwegian university ( N = 107) and compared with conventional grades using agreement statistics, discrepant-case expert review, examiner survey data, and judgment-time measures.

Results

ACJ produced high reliability (Scale Separation Reliability = 0.85–0.89) and substantial grade convergence (> 90% identical or adjacent grades), although fewer than half of grades matched exactly and course-specific shifts in grade outcomes were sensitive to boundary placement. Discrepant-case reviews suggested that surface-level features (structure and readability) were often favored over deeper analytical reasoning, consistent with examiner reflections and short median judgment times.

Discussion

Translating rank orders into grades required substantial post-hoc thresholding and coordination work, and the absence of individualized justifications constrained transparency. Overall, while ACJ can yield reliable rankings, its practical use in high-stakes settings involving long written responses raises important validity and feasibility concerns.

Zitieren

APA

Egelandsdal, K., & Færstad, J. (2026). Beyond reliability: examining the applicability of adaptive comparative judgment in high-stakes assessment. Frontiers in Education, 11. https://doi.org/10.3389/feduc.2026.1755046

Vancouver

Egelandsdal K, Færstad J. Beyond reliability: examining the applicability of adaptive comparative judgment in high-stakes assessment. Frontiers in Education. 2026;11. doi:10.3389/feduc.2026.1755046

BibTeX
@article{Egelandsdal2026,
  author = {Egelandsdal, Kjetil and Færstad, Jan-Ove},
  title = {Beyond reliability: examining the applicability of adaptive comparative judgment in high-stakes assessment},
  journal = {Frontiers in Education},
  year = {2026},
  volume = {11},
  doi = {10.3389/feduc.2026.1755046}
}
Kostenlos starten →