Knowledge Distillation and Structured Pruning at Matched Footprint for Sub-50 KB Cattle Behaviour Classification
Journal of Applied Science, Engineering and Technology for Development · 16.11.2026 · 6 (1)
DOI: 10.33803/kg23r7.2 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Applied Science, Engineering and Technology for Development, online erschienen am 16.11.2026. Verfasst von Bright Mutuma Gitari und 3 weiteren Autor:innen.
Abstract (Originalsprache)
On-collar behaviour classification underpins automated oestrus and health monitoring in cattle, but accurate models exceed the memory a low-cost collar can afford. Knowledge distillation and structured pruning are both proposed as the remedy, but on-collar studies test them separately and without the control that makes both interpretable: a same-size network trained from hard labels. Using two public neck-accelerometer datasets of cattle, 6 Japanese Black beef cows (the beef dataset) and 30 pre-weaned calves (the calf dataset), a convolutional teacher is trained under leave-one-animal-out validation and compressed along three routes at a matched int8 footprint, with the frontier re-derived on the calf dataset. First, the released beef dataset contains cross-animal duplication: all 15 animal pairs share byte-identical blocks of varying signal, and leaving it in place inflates macro F1 by +0.076 even when each animal is scored only on the classes it shows (p = 0.0007). Second, the capacity deficit is almost nil: a 1.72 KB network trained from hard labels reaches 0.681 macro F1 against the 127.1 KB teacher’s 0.692, which caps what compression can be worth. Within that ceiling, neither route is reliably better than same-size training. Structured pruning gains +0.0130 at one seed (p = 0.0096, n = 42) but −0.0209 to +0.0145 over five seeds, collapsing for some animals at the smallest widths at three seeds. Fine-tuning it on the same schedule as the other routes removes most collapses and the margin with them (−0.0080 to +0.0038 over three seeds, never significant). Distillation gains are +0.0021 to +0.0086, significant at one seed. Third, on calves, where shrinking costs up to +0.077, pruning falls below same-size training at five of five seeds, significantly at two, so no ordering here is a property of the methods.
Zitieren
APA
Gitari, B. M., Chiang, Y., Kiiru, C. W., & Kang'iri, S. M. (2026). Knowledge Distillation and Structured Pruning at Matched Footprint for Sub-50 KB Cattle Behaviour Classification. Journal of Applied Science, Engineering and Technology for Development, 6(1). https://doi.org/10.33803/kg23r7.2
Vancouver
Gitari BM, Chiang Y, Kiiru CW, Kang'iri SM. Knowledge Distillation and Structured Pruning at Matched Footprint for Sub-50 KB Cattle Behaviour Classification. Journal of Applied Science, Engineering and Technology for Development. 2026;6(1). doi:10.33803/kg23r7.2
BibTeX
@article{Gitari2026,
author = {Gitari, Bright Mutuma and Chiang, Ying-Hsuan and Kiiru, Consolata Wanjiru and Kang'iri, Samuel Macharia},
title = {Knowledge Distillation and Structured Pruning at Matched Footprint for Sub-50 KB Cattle Behaviour Classification},
journal = {Journal of Applied Science, Engineering and Technology for Development},
year = {2026},
volume = {6},
number = {1},
doi = {10.33803/kg23r7.2}
}
EUVIMED