EUVIMEDEUVIMED entdecken
Crossref

DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition

Hayat Ullah, Muhammad Ali Shafique, Abbas Khan, Arslan Munir

IEEE Transactions on Circuits and Systems for Video Technology · 2026-03 · 36 (3) · 2952-2967

DOI: 10.1109/tcsvt.2025.3615909 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-03. Verfasst von Hayat Ullah und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 89 Literaturangaben. Gefördert von: Air Force Office of Scientific Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The landscape of video recognition has undergone a significant transformation, shifting from traditional Convolutional Neural Networks (CNNs) to Transformer-based architectures in order to achieve better accuracy. While CNNs, especially 3D variants, have excelled in capturing spatiotemporal dynamics for action recognition, recent developments in Transformer models, with their self-attention mechanisms, have proven highly effective in modeling long-range dependencies across space and time. Despite their state-of-the-art performance on prominent video recognition benchmarks, the computational demands of Transformers, particularly in processing dense video data, remain a significant hurdle. To address these challenges, we introduce a lightweight Video Focal Modulation Network named DVFL-Net, which distills the spatio-temporal knowledge from a large pre-trained teacher to nano student model, making it well-suited for on-device applications. By leveraging knowledge distillation and spatial-temporal feature extraction, our model significantly reduces computational overhead (approximately $7\times $ ) while maintaining high performance on video recognition tasks. We combine the forward Kullback–Leibler (KL) divergence and spatio-temporal focal modulation to distill the local and global spatio-temporal context from the Video-FocalNet Base (teacher) to our proposed nano VFL-Net (student) model. We extensively evaluate our DVFL-Net, both with and without forward KL divergence, against recent state-of-the-art HAR approaches on UCF50, UCF101, HMDB51, SSV2 and Kinetics-400 datasets. Further, we conducted a detailed ablation study in forward KL divergence settings and report the obtained observations. The obtained results confirm the superiority of the distilled VFL-Net (i.e., DVFL-Net) over existing methods, highlighting its optimal tradeoff between performance and computational efficiency, including reduced memory usage and lower GFLOPs, making it a highly efficient solution for HAR tasks. https://github.com/iscaas/DVFL-Net

Förderung

  • Air Force Office of Scientific Research

Zitieren

APA

Ullah, H., Shafique, M. A., Khan, A., & Munir, A. (2026). DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition. IEEE Transactions on Circuits and Systems for Video Technology, 36(3), 2952-2967. https://doi.org/10.1109/tcsvt.2025.3615909

Vancouver

Ullah H, Shafique MA, Khan A, Munir A. DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(3):2952-2967. doi:10.1109/tcsvt.2025.3615909

BibTeX
@article{Ullah2026,
  author = {Ullah, Hayat and Shafique, Muhammad Ali and Khan, Abbas and Munir, Arslan},
  title = {DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2026},
  volume = {36},
  number = {3},
  pages = {2952--2967},
  doi = {10.1109/tcsvt.2025.3615909}
}
Kostenlos starten →