EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Hybrid Knowledge Distillation for Edge-Efficient Video Action Recognition: Improving Lightweight 3D CNNs via Joint Distillation

Mohammad Rasras, Iuliana Marin

Computers · 05.06.2026 · 15 (6) · 371

DOI: 10.3390/computers15060371 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computers, online erschienen am 05.06.2026. Verfasst von Mohammad Rasras und 1 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 79 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: PubArt program of National University of Science and Technology POLITEHNICA Bucharest.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

One of the remaining challenges in deploying 3D CNN models in resource-constrained environments is the high computational demand. In this paper, we design three lightweight architectures that have distinct spatiotemporal topologies, namely, Lite-R21D, Lite-MC3, and Lite-LF, to reduce computational cost. However, these compact models have restricted representational capacity, which consequently limits their ability to capture complex spatiotemporal features. To overcome this, we employ Knowledge Distillation (KD) and further investigate hybrid combinations of response-based, spatiotemporal attention, and intermediate feature alignment paradigms. By analyzing knowledge transfer across these diverse architectures, our experiments on UCF101 and HMDB51 demonstrate that combining these distillation configurations consistently outperforms single KD methods, resulting in a substantial increase in accuracy across all Student models. Our optimal hybrid setup achieves 92.07% accuracy on UCF101 and 65.56% on HMDB51, compared to the Teacher’s 94.74% and 69.48%, reducing the accuracy gap to only 2.67% and 3.92%. These gains are achieved alongside significant efficiency improvements. The proposed models operate with up to 87% fewer parameters and an 89% reduction in Floating-Point Operations (FLOPs), achieving 6.7× faster inference. Our findings highlight that hybrid distillation is an effective approach for transferring and utilizing complex spatiotemporal knowledge in lightweight models.

Förderung

  • PubArt program of National University of Science and Technology POLITEHNICA Bucharest

Zitieren

APA

Rasras, M., & Marin, I. (2026). Hybrid Knowledge Distillation for Edge-Efficient Video Action Recognition: Improving Lightweight 3D CNNs via Joint Distillation. Computers, 15(6), 371. https://doi.org/10.3390/computers15060371

Vancouver

Rasras M, Marin I. Hybrid Knowledge Distillation for Edge-Efficient Video Action Recognition: Improving Lightweight 3D CNNs via Joint Distillation. Computers. 2026;15(6):371. doi:10.3390/computers15060371

BibTeX
@article{Rasras2026,
  author = {Rasras, Mohammad and Marin, Iuliana},
  title = {Hybrid Knowledge Distillation for Edge-Efficient Video Action Recognition: Improving Lightweight 3D CNNs via Joint Distillation},
  journal = {Computers},
  year = {2026},
  volume = {15},
  number = {6},
  pages = {371},
  doi = {10.3390/computers15060371}
}
Kostenlos starten →