Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology
Veterinary Ophthalmology · 2026-09 · 29 (5)
DOI: 10.1111/vop.70208 ↗Kurz gefasst
Fachartikel in der Zeitschrift Veterinary Ophthalmology, online erschienen am 19.07.2026. Verfasst von Malwina Ewa Kowalska und 4 weiteren Autor:innen. Beteiligte Einrichtung: Ophthalmology Section, Vetsuisse Faculty, University of Zurich, Zurich, Switzerland. Hauptthemen laut MEDLINE: Dog Diseases, Eye Diseases, Ophthalmology. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Objective
Cohen's kappa is widely used to measure rater agreement on binary outcomes and performs well when outcome prevalence is relatively balanced. This paper aims to explain the challenges encountered with Cohen's kappa, and to provide guidance to veterinary ophthalmologists on the usage and interpretation of kappa values. Animals Studied Article based on an existing dataset in which two examiners graded iridocorneal angle abnormalities in 60 client‐owned dogs during pre‐breeding examinations. Procedure Following the 2022 ECVO‐HED (European College of Veterinary Ophthalmology—Hereditary Eye Diseases) gonioscopy grading scheme and its general recommendations, eyes were classified into two categories—“breeding‐YES” (not affected, mildly or moderately affected) or “breeding‐NO” (severely affected). Inter‐examiner agreement was separately assessed for real (60 left and 58 right eyes) and simulated data (manually balanced left eye dataset with unchanged agreement between examiners).
Results
While the two examiners classified 52/60 left and 51/58 right eyes into the “breeding‐YES” category, they disagreed on 7/60 left and 1/58 right eyes, with resulting kappa values of 0.18 and 0.91 for the left and right eyes, respectively. After balancing the data, with both examiners classifying 27/60 left eyes into the “breeding‐yes” category, but still disagreeing on 7/60 eyes, the kappa value increased to 0.77.
Conclusions
Readers should be cautious when comparing kappa values across studies with different underlying prevalence and bias. Under non‐ideal data distribution, additional statistical indices like prevalence‐ and bias index, prevalence‐and‐bias‐adjusted kappa (PABAK), and maxKappa, do not replace Cohen's kappa, but assist in interpreting it.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Ophthalmology Section, Vetsuisse Faculty, University of Zurich, Zurich, Switzerland.
- Section of Epidemiology, Vetsuisse Faculty, University of Zurich, Zurich, Switzerland.
- Centre for Veterinary Systems Transformation and Sustainability, Clinical Department for Farm Animals and Food System Science, University of Veterinary Medicine Vienna, Vienna, Austria.
ORCID-Profile: Malwina Ewa Kowalska, Niklas Holz, Simon Anton Pot, Antonella Rampazzo, Sonja Hartnack
Zitieren
APA
Kowalska, M. E., Holz, N., Pot, S. A., Rampazzo, A., & Hartnack, S. (2026). Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology. Veterinary Ophthalmology, 29(5). https://doi.org/10.1111/vop.70208
Vancouver
Kowalska ME, Holz N, Pot SA, Rampazzo A, Hartnack S. Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology. Veterinary Ophthalmology. 2026;29(5). doi:10.1111/vop.70208
BibTeX
@article{Kowalska2026,
author = {Kowalska, Malwina Ewa and Holz, Niklas and Pot, Simon Anton and Rampazzo, Antonella and Hartnack, Sonja},
title = {Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology},
journal = {Veterinary Ophthalmology},
year = {2026},
volume = {29},
number = {5},
doi = {10.1111/vop.70208}
}
EUVIMED