The state and expectations of interrater reliability in nonhuman personality research.
Journal of Comparative Psychology · 2026
DOI: 10.1037/com0000439 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Comparative Psychology, 2026. Verfasst von Alexander J Pritchard. Beteiligte Einrichtung: Neuroscience and Behavior Unit, California National Primate Research Center, University of California, Davis. Hauptthemen laut MEDLINE: Personality, Behavior, Animal, Primates.
Abstract (Originalsprache)
Establishing interobserver reliability is a key step for asserting concordance in personality ratings. Yet, typical measures of reliability-such as intraclass correlation coefficients (ICCs)-are not just reflective of whether raters agree on subject variation. Rather, ICC estimates can be influenced by a multitude of data qualities such that high reliability need not demonstrate validity. I explored what our expectations are for ICCs based on the state of published studies' estimates and based on simulations built upon the typical properties of Likert ratings used for nonhuman assessments. I relied on the nonhuman primate personality literature, assembling ICC(3, 1) estimates from 34 publications. I conducted Bayesian meta-analyses, which demonstrate that ICC(3, 1) estimates showed a posterior estimate of 0.31, but that items exhibited high variability. This variability showed concordance with the underlying traits and dimensions, as well as taxonomic distinctions. Simulations, however, demonstrated the complexities of using ICC estimates as a measure of reliability. Furthermore, the simulations showed that neat threshold cutoffs are of poor utility without consideration of the properties of the data and system, namely, the number of raters and subjects, as well as distributions of the item scores. I provide recommendations based on expectations of these data. Nonhuman primate ICC estimates show reasonable values relative to expectations of reasonable true subject variation, in line with repeatability estimations from behavior, physiology, and expectations from human individual differences. (PsycInfo Database Record (c) 2026 APA, all rights reserved).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Neuroscience and Behavior Unit, California National Primate Research Center, University of California, Davis.
ORCID-Profile: Alexander J Pritchard
Zitieren
APA
Pritchard, A. J. (2026). The state and expectations of interrater reliability in nonhuman personality research. Journal of Comparative Psychology. https://doi.org/10.1037/com0000439
Vancouver
Pritchard AJ. The state and expectations of interrater reliability in nonhuman personality research. Journal of Comparative Psychology. 2026. doi:10.1037/com0000439
BibTeX
@article{Pritchard2026,
author = {Pritchard, Alexander J},
title = {The state and expectations of interrater reliability in nonhuman personality research},
journal = {Journal of Comparative Psychology},
year = {2026},
doi = {10.1037/com0000439}
}
EUVIMED