How reliable is visual lameness scoring? Assessing human label variability for use in automated detection systems
Journal of Dairy Research · 2026
DOI: 10.1017/s0022029926102180 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Dairy Research, 2026. Verfasst von Konstantina Linardopoulou und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Biodiversity, One Health and Veterinary Medicine, University of Glasgow, Glasgow, UK. Hauptthemen laut MEDLINE: Lameness, Animal, Cattle Diseases. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Visual mobility scoring to detect lame dairy cattle can be subjective and inconsistent. This study assessed the reliability of visual mobility scores from multiple assessors, using different scoring methods (live vs. video) and experience levels to evaluate their influence on label quality for machine learning applications. We gathered data from two farms using the AHDB 4-point mobility scale and a simplified post-hoc dichotomised version, with both live and video assessments. Substantial within- and between-assessor variation was seen in scores, particularly for scores 0 and 1 (consistent with normal and slightly abnormal gaits, respectively). Assessors showed only fair (weighted kappa ≈ 0.33) score consistency when they scored the same animal in different ways (live vs. video). Post-hoc simplification of the four-level scores to a dichotomous score improved agreement but reduced granularity. Assessor experience had limited influence on agreement levels ( P > 0.05), and increased video viewing frequency during the assessment process was associated with lower inter-assessor agreement (probability estimate = −0.49, P = 0.005), suggesting higher uncertainty in ambiguous cases. Qualitative feedback from assessor comments revealed that the speed of the animal affected their scoring decisions (β = –1.92, P = 0.007). These results highlight the difficulties in using subjective human scores as labels for machine learning training. To improve automatic lameness detection in dairy cattle, we need strategies to reduce this variation and use more definitive labels.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Biodiversity, One Health and Veterinary Medicine, University of Glasgow, Glasgow, UK.
- James Watt School of Engineering, University of Glasgow, Glasgow, UK.
ORCID-Profile: Nicholas N Jonsson
Zitieren
APA
Linardopoulou, K., Viora, L., King, G., Le Kernec, J., & Jonsson, N. N. (2026). How reliable is visual lameness scoring? Assessing human label variability for use in automated detection systems. Journal of Dairy Research. https://doi.org/10.1017/s0022029926102180
Vancouver
Linardopoulou K, Viora L, King G, Le Kernec J, Jonsson NN. How reliable is visual lameness scoring? Assessing human label variability for use in automated detection systems. Journal of Dairy Research. 2026. doi:10.1017/s0022029926102180
BibTeX
@article{Linardopoulou2026,
author = {Linardopoulou, Konstantina and Viora, Lorenzo and King, George and Le Kernec, Julien and Jonsson, Nicholas N},
title = {How reliable is visual lameness scoring? Assessing human label variability for use in automated detection systems},
journal = {Journal of Dairy Research},
year = {2026},
doi = {10.1017/s0022029926102180}
}
EUVIMED