Learning Multi-View Interactional Skeleton Graph for Action Recognition
IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.06.2023 · 45 (6) · 6940-6954
DOI: 10.1109/tpami.2020.3032738 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.06.2023. Verfasst von Minsi Wang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 55-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Science and Technology Commission of Shanghai Municipality.
Abstract (Originalsprache)
Capturing the interactions of human articulations lies in the center of skeleton-based action recognition. Recent graph-based methods are inherently limited in the weak spatial context modeling capability due to fixed interaction pattern and inflexible shared weights of GCN. To address above problems, we propose the multi-view interactional graph network (MV-IGNet) which can construct, learn and infer multi-level spatial skeleton context, including view-level (global), group-level, joint-level (local) context, in a unified way. MV-IGNet leverages different skeleton topologies as multi-views to cooperatively generate complementary action features. For each view, separable parametric graph convolution (SPG-Conv) enables multiple parameterized graphs to enrich local interaction patterns, which provides strong graph-adaption ability to handle irregular skeleton topologies. We also partition the skeleton into several groups and then the higher-level group contexts including inter-group and intra-group, are hierarchically captured by above SPG-Conv layers. A simple yet effective global context adaption (GCA) module facilitates representative feature extraction by learning the input-dependent skeleton topologies. Compared to the mainstream works, MV-IGNet can be readily implemented while with smaller model size and faster inference. Experimental results show the proposed MV-IGNet achieves impressive performance on large-scale benchmarks: NTU-RGB+D and NTU-RGB+D 120.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Science and Technology Commission of Shanghai Municipality
- Open Project Program
- Nanjing University of Information Science & Technology
Zitieren
APA
Wang, M., Ni, B., & Yang, X. (2023). Learning Multi-View Interactional Skeleton Graph for Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6), 6940-6954. https://doi.org/10.1109/tpami.2020.3032738
Vancouver
Wang M, Ni B, Yang X. Learning Multi-View Interactional Skeleton Graph for Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(6):6940-6954. doi:10.1109/tpami.2020.3032738
BibTeX
@article{Wang2023,
author = {Wang, Minsi and Ni, Bingbing and Yang, Xiaokang},
title = {Learning Multi-View Interactional Skeleton Graph for Action Recognition},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2023},
volume = {45},
number = {6},
pages = {6940--6954},
doi = {10.1109/tpami.2020.3032738}
}
EUVIMED