Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition
IEEE Transactions on Circuits and Systems for Video Technology · 2023-04 · 33 (4) · 1868-1883
DOI: 10.1109/tcsvt.2022.3217763 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-04. Verfasst von Zengxi Huang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 60-mal zitiert (Crossref) und nennt 83 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Sichuan Science and Technology Program, Xihua University Funds for Young Scholars.
Abstract (Originalsprache)
Graph convolutional networks (GCN) have attracted increasing interest in action recognition in recent years. GCN models human skeleton sequences as spatio-temporal graphs. Also, attention mechanisms are often jointly used with GCNs to highlight important frames or body joints in a sequence. However, attention modules learn parameters offline and are fixed, so may not adapt well to unseen samples. In this paper, we propose a simple but effective motion-driven spatial and temporal adaptation strategy to dynamically strengthen the features of important frames and joints for skeleton-based action recognition. The rationale is that the joints and frames with dramatic motions are generally more informative and discriminative. We combine the spatial and temporal refinements by using a two-branch structure, in which the joint and frame-wise feature refinements perform in parallel. Such a structure can lead to learn more complementary feature representations. Moreover, we propose to use the fully connected graph convolution to learn the long-range spatial dependencies. Besides, we investigate two high-resolution skeleton graphs by creating virtual joints, aiming to improve the representation of skeleton features. By combining the above proposals, we develop a novel motion-driven spatial and temporal adaptive high-resolution GCN. Experimental results demonstrate that the proposed model achieves state-of-the-art (SOTA) results on the challenging large-scale Kinetics-Skeleton and UAV-Human datasets, and it is on par with the SOTA methods on the two NTU-RGB+D 60&120 datasets. Additionally, our motion-driven adaptation method shows encouraging performance when compared with the attention mechanisms.
Förderung
- National Natural Science Foundation of China
- Sichuan Science and Technology Program
- Xihua University Funds for Young Scholars
- Sichuan University
Zitieren
APA
Huang, Z., Qin, Y., Lin, X., Liu, T., Feng, Z., & Liu, Y. (2023). Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition. IEEE Transactions on Circuits and Systems for Video Technology, 33(4), 1868-1883. https://doi.org/10.1109/tcsvt.2022.3217763
Vancouver
Huang Z, Qin Y, Lin X, Liu T, Feng Z, Liu Y. Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(4):1868-1883. doi:10.1109/tcsvt.2022.3217763
BibTeX
@article{Huang2023,
author = {Huang, Zengxi and Qin, Yusong and Lin, Xiaobing and Liu, Tianlin and Feng, Zhenhua and Liu, Yiguang},
title = {Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2023},
volume = {33},
number = {4},
pages = {1868--1883},
doi = {10.1109/tcsvt.2022.3217763}
}
EUVIMED