Human action recognition in immersive virtual reality based on multi‐scale spatio‐temporal attention network
Computer Animation and Virtual Worlds · 2024-09 · 35 (5)
DOI: 10.1002/cav.2293 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computer Animation and Virtual Worlds, online erschienen am 23.09.2024. Verfasst von Zhiyong Xiao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 79-mal zitiert (Crossref) und nennt 51 Literaturangaben. Gefördert von: China Scholarship Council, National Natural Science Foundation of China.
Abstract (Originalsprache)
Wearable human action recognition (HAR) has practical applications in daily life. However, traditional HAR methods solely focus on identifying user movements, lacking interactivity and user engagement. This paper proposes a novel immersive HAR method called MovPosVR. Virtual reality (VR) technology is employed to create realistic scenes and enhance the user experience. To improve the accuracy of user action recognition in immersive HAR, a multi‐scale spatio‐temporal attention network (MSSTANet) is proposed. The network combines the convolutional residual squeeze and excitation (CRSE) module with the multi‐branch convolution and long short‐term memory (MCLSTM) module to extract spatio‐temporal features and automatically select relevant features from action signals. Additionally, a multi‐head attention with shared linear mechanism (MHASLM) module is designed to facilitate information interaction, further enhancing feature extraction and improving accuracy. The MSSTANet network achieves superior performance, with accuracy rates of 99.33% and 98.83% on the publicly available WISDM and PAMPA2 datasets, respectively, surpassing state‐of‐the‐art networks. Our method showcases the potential to display user actions and position information in a virtual world, enriching user experiences and interactions across diverse application scenarios.
Förderung
- China Scholarship Council
- National Natural Science Foundation of China
Zitieren
APA
Xiao, Z., Chen, Y., Zhou, X., He, M., Liu, L., Yu, F., & Jiang, M. (2024). Human action recognition in immersive virtual reality based on multi‐scale spatio‐temporal attention network. Computer Animation and Virtual Worlds, 35(5). https://doi.org/10.1002/cav.2293
Vancouver
Xiao Z, Chen Y, Zhou X, He M, Liu L, Yu F, et al. Human action recognition in immersive virtual reality based on multi‐scale spatio‐temporal attention network. Computer Animation and Virtual Worlds. 2024;35(5). doi:10.1002/cav.2293
BibTeX
@article{Xiao2024,
author = {Xiao, Zhiyong and Chen, Yukun and Zhou, Xinlei and He, Mingwei and Liu, Li and Yu, Feng and Jiang, Minghua},
title = {Human action recognition in immersive virtual reality based on multi‐scale spatio‐temporal attention network},
journal = {Computer Animation and Virtual Worlds},
year = {2024},
volume = {35},
number = {5},
doi = {10.1002/cav.2293}
}
EUVIMED