EUVIMEDEUVIMED entdecken
Crossref

EPAM-Net: An efficient pose-driven attention-guided multimodal network for video action recognition

Ahmed Abdelkawy, Asem Ali, Aly Farag

Neurocomputing · 2025-06 · 633 · 129781

DOI: 10.1016/j.neucom.2025.129781 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Neurocomputing, 2025-06. Verfasst von Ahmed Abdelkawy und 2 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 45 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Existing multimodal-based human action recognition approaches are computationally intensive, limiting their deployment in real-time applications. In this work, we present a novel and efficient pose-driven attention-guided multimodal network (EPAM-Net) for action recognition in videos. Specifically, we propose eXpand temporal Shift (X-ShiftNet) convolutional architectures for RGB and pose streams to capture spatio-temporal features from RGB videos and their skeleton sequences. The X-ShiftNet tackles the high computational cost of the 3D CNNs by integrating the Temporal Shift Module (TSM) into an efficient 2D CNN, enabling efficient spatiotemporal learning. Then skeleton features are utilized to guide the visual network stream, focusing on keyframes and their salient spatial regions using the proposed spatial-temporal attention block. Finally, the predictions of the two streams are fused for final classification. The experimental results show that our method, with a significant reduction in floating-point operations (FLOPs), outperforms and competes with the state-of-the-art methods on NTU RGB-D 60, NTU RGB-D 120, PKU-MMD, and Toyota SmartHome datasets. The proposed EPAM-Net provides up to a 72.8x reduction in FLOPs and up to a 48.6x reduction in the number of network parameters. The code will be available at https://github.com/ahmed-nady/Multimodal-Action-Recognition.

Zitieren

APA

Abdelkawy, A., Ali, A., & Farag, A. (2025). EPAM-Net: An efficient pose-driven attention-guided multimodal network for video action recognition. Neurocomputing, 633, 129781. https://doi.org/10.1016/j.neucom.2025.129781

Vancouver

Abdelkawy A, Ali A, Farag A. EPAM-Net: An efficient pose-driven attention-guided multimodal network for video action recognition. Neurocomputing. 2025;633:129781. doi:10.1016/j.neucom.2025.129781

BibTeX
@article{Abdelkawy2025,
  author = {Abdelkawy, Ahmed and Ali, Asem and Farag, Aly},
  title = {EPAM-Net: An efficient pose-driven attention-guided multimodal network for video action recognition},
  journal = {Neurocomputing},
  year = {2025},
  volume = {633},
  pages = {129781},
  doi = {10.1016/j.neucom.2025.129781}
}
Kostenlos starten →