EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology

Malwina Ewa Kowalska iD, Niklas Holz iD, Simon Anton Pot iD, Antonella Rampazzo iD, Sonja Hartnack iD

Veterinary Ophthalmology · 2026-09 · 29 (5)

DOI: 10.1111/vop.70208 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Veterinary Ophthalmology, online erschienen am 19.07.2026. Verfasst von Malwina Ewa Kowalska und 4 weiteren Autor:innen. Beteiligte Einrichtung: Ophthalmology Section, Vetsuisse Faculty, University of Zurich, Zurich, Switzerland. Hauptthemen laut MEDLINE: Dog Diseases, Eye Diseases, Ophthalmology. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Cohen's kappa is widely used to measure rater agreement on binary outcomes and performs well when outcome prevalence is relatively balanced. This paper aims to explain the challenges encountered with Cohen's kappa, and to provide guidance to veterinary ophthalmologists on the usage and interpretation of kappa values. Animals Studied Article based on an existing dataset in which two examiners graded iridocorneal angle abnormalities in 60 client‐owned dogs during pre‐breeding examinations. Procedure Following the 2022 ECVO‐HED (European College of Veterinary Ophthalmology—Hereditary Eye Diseases) gonioscopy grading scheme and its general recommendations, eyes were classified into two categories—“breeding‐YES” (not affected, mildly or moderately affected) or “breeding‐NO” (severely affected). Inter‐examiner agreement was separately assessed for real (60 left and 58 right eyes) and simulated data (manually balanced left eye dataset with unchanged agreement between examiners).

Results

While the two examiners classified 52/60 left and 51/58 right eyes into the “breeding‐YES” category, they disagreed on 7/60 left and 1/58 right eyes, with resulting kappa values of 0.18 and 0.91 for the left and right eyes, respectively. After balancing the data, with both examiners classifying 27/60 left eyes into the “breeding‐yes” category, but still disagreeing on 7/60 eyes, the kappa value increased to 0.77.

Conclusions

Readers should be cautious when comparing kappa values across studies with different underlying prevalence and bias. Under non‐ideal data distribution, additional statistical indices like prevalence‐ and bias index, prevalence‐and‐bias‐adjusted kappa (PABAK), and maxKappa, do not replace Cohen's kappa, but assist in interpreting it.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Dog DiseasesEye DiseasesOphthalmologyAnimalsDogsObserver VariationGonioscopy

Stichwörter der Autor:innen

gonioscopyinter‐raterkappa coefficientmaxKappareliability studies

Autor:innen und Institutionen

  • Ophthalmology Section, Vetsuisse Faculty, University of Zurich, Zurich, Switzerland.
  • Section of Epidemiology, Vetsuisse Faculty, University of Zurich, Zurich, Switzerland.
  • Centre for Veterinary Systems Transformation and Sustainability, Clinical Department for Farm Animals and Food System Science, University of Veterinary Medicine Vienna, Vienna, Austria.

ORCID-Profile: Malwina Ewa Kowalska, Niklas Holz, Simon Anton Pot, Antonella Rampazzo, Sonja Hartnack

Zitieren

APA

Kowalska, M. E., Holz, N., Pot, S. A., Rampazzo, A., & Hartnack, S. (2026). Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology. Veterinary Ophthalmology, 29(5). https://doi.org/10.1111/vop.70208

Vancouver

Kowalska ME, Holz N, Pot SA, Rampazzo A, Hartnack S. Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology. Veterinary Ophthalmology. 2026;29(5). doi:10.1111/vop.70208

BibTeX
@article{Kowalska2026,
  author = {Kowalska, Malwina Ewa and Holz, Niklas and Pot, Simon Anton and Rampazzo, Antonella and Hartnack, Sonja},
  title = {Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology},
  journal = {Veterinary Ophthalmology},
  year = {2026},
  volume = {29},
  number = {5},
  doi = {10.1111/vop.70208}
}
Kostenlos starten →