EUVIMEDEUVIMED entdecken
Crossref

Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition

Zengxi Huang, Yusong Qin, Xiaobing Lin, Tianlin Liu, Zhenhua Feng, Yiguang Liu

IEEE Transactions on Circuits and Systems for Video Technology · 2023-04 · 33 (4) · 1868-1883

DOI: 10.1109/tcsvt.2022.3217763 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-04. Verfasst von Zengxi Huang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 60-mal zitiert (Crossref) und nennt 83 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Sichuan Science and Technology Program, Xihua University Funds for Young Scholars.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Graph convolutional networks (GCN) have attracted increasing interest in action recognition in recent years. GCN models human skeleton sequences as spatio-temporal graphs. Also, attention mechanisms are often jointly used with GCNs to highlight important frames or body joints in a sequence. However, attention modules learn parameters offline and are fixed, so may not adapt well to unseen samples. In this paper, we propose a simple but effective motion-driven spatial and temporal adaptation strategy to dynamically strengthen the features of important frames and joints for skeleton-based action recognition. The rationale is that the joints and frames with dramatic motions are generally more informative and discriminative. We combine the spatial and temporal refinements by using a two-branch structure, in which the joint and frame-wise feature refinements perform in parallel. Such a structure can lead to learn more complementary feature representations. Moreover, we propose to use the fully connected graph convolution to learn the long-range spatial dependencies. Besides, we investigate two high-resolution skeleton graphs by creating virtual joints, aiming to improve the representation of skeleton features. By combining the above proposals, we develop a novel motion-driven spatial and temporal adaptive high-resolution GCN. Experimental results demonstrate that the proposed model achieves state-of-the-art (SOTA) results on the challenging large-scale Kinetics-Skeleton and UAV-Human datasets, and it is on par with the SOTA methods on the two NTU-RGB+D 60&120 datasets. Additionally, our motion-driven adaptation method shows encouraging performance when compared with the attention mechanisms.

Förderung

  • National Natural Science Foundation of China
  • Sichuan Science and Technology Program
  • Xihua University Funds for Young Scholars
  • Sichuan University

Zitieren

APA

Huang, Z., Qin, Y., Lin, X., Liu, T., Feng, Z., & Liu, Y. (2023). Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition. IEEE Transactions on Circuits and Systems for Video Technology, 33(4), 1868-1883. https://doi.org/10.1109/tcsvt.2022.3217763

Vancouver

Huang Z, Qin Y, Lin X, Liu T, Feng Z, Liu Y. Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(4):1868-1883. doi:10.1109/tcsvt.2022.3217763

BibTeX
@article{Huang2023,
  author = {Huang, Zengxi and Qin, Yusong and Lin, Xiaobing and Liu, Tianlin and Feng, Zhenhua and Liu, Yiguang},
  title = {Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2023},
  volume = {33},
  number = {4},
  pages = {1868--1883},
  doi = {10.1109/tcsvt.2022.3217763}
}
Kostenlos starten →