EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

3D-ShuffleViT: An Efficient Video Action Recognition Network with Deep Integration of Self-Attention and Convolution

Yinghui Wang, Anlei Zhu, Haomiao Ma, Lingyu Ai, Wei Song, Shaojie Zhang

Mathematics · 08.09.2023 · 11 (18) · 3848

DOI: 10.3390/math11183848 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematics, online erschienen am 08.09.2023. Verfasst von Yinghui Wang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, “Double Creation” Plan of Jiangsu Province, “Taihu Talent-Innovative Leading Talent” Plan of Wuxi City.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Compared with traditional methods, the action recognition model based on 3D convolutional deep neural network captures spatio-temporal features more accurately, resulting in higher accuracy. However, the large number of parameters and computational requirements of 3D models make it difficult to deploy on mobile devices with limited computing power. In order to achieve an efficient video action recognition model, we have analyzed and compared classic lightweight network principles and proposed the 3D-ShuffleViT network. By deeply integrating the self-attention mechanism with convolution, we have introduced an efficient ACISA module that further enhances the performance of our proposed model. This has resulted in exceptional performance in both context-sensitive and context-independent action recognition, while reducing deployment costs. It is worth noting that our 3D-ShuffleViT network, with a computational cost of only 6% of that of SlowFast-ResNet101, achieved 98% of the latter’s Top1 accuracy on the EgoGesture dataset. Furthermore, on the same CPU (Intel i5-8300H), its speed was 2.5 times that of the latter. In addition, when we deployed our model on edge devices, our proposed network achieved the best balance between accuracy and speed among lightweight networks of the same order.

Förderung

  • National Natural Science Foundation of China
  • “Double Creation” Plan of Jiangsu Province
  • “Taihu Talent-Innovative Leading Talent” Plan of Wuxi City

Zitieren

APA

Wang, Y., Zhu, A., Ma, H., Ai, L., Song, W., & Zhang, S. (2023). 3D-ShuffleViT: An Efficient Video Action Recognition Network with Deep Integration of Self-Attention and Convolution. Mathematics, 11(18), 3848. https://doi.org/10.3390/math11183848

Vancouver

Wang Y, Zhu A, Ma H, Ai L, Song W, Zhang S. 3D-ShuffleViT: An Efficient Video Action Recognition Network with Deep Integration of Self-Attention and Convolution. Mathematics. 2023;11(18):3848. doi:10.3390/math11183848

BibTeX
@article{Wang2023,
  author = {Wang, Yinghui and Zhu, Anlei and Ma, Haomiao and Ai, Lingyu and Song, Wei and Zhang, Shaojie},
  title = {3D-ShuffleViT: An Efficient Video Action Recognition Network with Deep Integration of Self-Attention and Convolution},
  journal = {Mathematics},
  year = {2023},
  volume = {11},
  number = {18},
  pages = {3848},
  doi = {10.3390/math11183848}
}
Kostenlos starten →