Pose-Guided Robust Action Recognition for Outdoor Internet of Things
IEEE Transactions on Consumer Electronics · 2024-11 · 70 (4) · 7032-7043
DOI: 10.1109/tce.2024.3384974 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Consumer Electronics, 2024-11. Verfasst von Jiahui Yu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 67 Literaturangaben. Gefördert von: Zhejiang Provincial Natural Science Foundation of China, Autism Research Special Fund of Zhejiang Foundation for Disabled Persons, Cao Guangbiao High-Tech Development Fund.
Abstract (Originalsprache)
Skeleton-based human recognition is a key technology for visual feedback, which can help the Internet of Things (IoT) interact with humans in a non-contact manner outdoors. Graph Convolutional Networks (GCNs), achieving an intuitive understanding of the human skeleton, have received increasing attention. Although current GCN-based works explore how to model unlinked body parts, they still show weak robustness in noisy solid data, such as joint/frame loss, which often happens in outdoor IoTs. Towards robust visual feedback, in the paper, we propose robust skeleton-based action recognition neural networks (Robust-SAR), a new cost-efficient approach for recognizing activities in noisy outdoor scenarios. Instead of estimating 2D or 3D skeleton coordinates, we first extract the heatmap of human poses from videos. We propose S-pose to learn heatmap at multiple levels, i.e., joint, joint-scale, and part-scale learning, boosting higher-order motion pattern learning. Additionally, we propose T-pose to adaptively employ the features of previous frames to enhance the current frame, further enhancing the robustness of spatiotemporal human representation. Experimentally, Robust-SAR achieves state-of-the-art recognition results on four benchmarks, including NTU-60, NTU-120, NUCLA, and Kinetics-400 (outdoor datasets). Furthermore, in noise-filled outdoor conditions, the performance of Robust-SAR only drops by about 0.5%, while other state-of-the-art methods drop by about 2%.
Förderung
- Zhejiang Provincial Natural Science Foundation of China
- Autism Research Special Fund of Zhejiang Foundation for Disabled Persons
- Cao Guangbiao High-Tech Development Fund
- Fundamental Research Funds for the Central Universities
Zitieren
APA
Yu, J., Cheng, X., Chen, H., & Xu, Y. (2024). Pose-Guided Robust Action Recognition for Outdoor Internet of Things. IEEE Transactions on Consumer Electronics, 70(4), 7032-7043. https://doi.org/10.1109/tce.2024.3384974
Vancouver
Yu J, Cheng X, Chen H, Xu Y. Pose-Guided Robust Action Recognition for Outdoor Internet of Things. IEEE Transactions on Consumer Electronics. 2024;70(4):7032-7043. doi:10.1109/tce.2024.3384974
BibTeX
@article{Yu2024,
author = {Yu, Jiahui and Cheng, Xu and Chen, Hang and Xu, Yingke},
title = {Pose-Guided Robust Action Recognition for Outdoor Internet of Things},
journal = {IEEE Transactions on Consumer Electronics},
year = {2024},
volume = {70},
number = {4},
pages = {7032--7043},
doi = {10.1109/tce.2024.3384974}
}
EUVIMED