MAF-Net: A multimodal data fusion approach for human action recognition
PLOS ONE · 09.04.2025 · 20 (4) · e0319656
DOI: 10.1371/journal.pone.0319656 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 09.04.2025. Verfasst von Dongwei Xie und 4 weiteren Autor:innen. Beteiligte Einrichtung: Guangdong Engineering Polytechnic, Guangzhou, China. Hauptthemen laut MEDLINE: Pattern Recognition, Automated. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 73 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
3D skeleton-based human activity recognition has gained significant attention due to its robustness against variations in background, lighting, and viewpoints. However, challenges remain in effectively capturing spatiotemporal dynamics and integrating complementary information from multiple data modalities, such as RGB video and skeletal data. To address these challenges, we propose a multimodal fusion framework that leverages optical flow-based key frame extraction, data augmentation techniques, and an innovative fusion of skeletal and RGB streams using self-attention and skeletal attention modules. The model employs a late fusion strategy to combine skeletal and RGB features, allowing for more effective capture of spatial and temporal dependencies. Extensive experiments on benchmark datasets, including NTU RGB+D, SYSU, and UTD-MHAD, demonstrate that our method outperforms existing models. This work not only enhances action recognition accuracy but also provides a robust foundation for future multimodal integration and real-time applications in diverse fields such as surveillance and healthcare.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Guangdong Engineering Polytechnic, Guangzhou, China.
- Guangdong Eco-Engineering Polytechnic, Guangzhou, China.
- Guangdong Justice Police Vocational College, Guangzhou, China.
- Zhongkai University of Agriculture and Engineering, Guangzhou, China.
ORCID-Profile: Xiang Gao
Zitieren
APA
Xie, D., Zhang, X., Gao, X., Zhao, H., & Du, D. (2025). MAF-Net: A multimodal data fusion approach for human action recognition. PLOS ONE, 20(4), e0319656. https://doi.org/10.1371/journal.pone.0319656
Vancouver
Xie D, Zhang X, Gao X, Zhao H, Du D. MAF-Net: A multimodal data fusion approach for human action recognition. PLOS ONE. 2025;20(4):e0319656. doi:10.1371/journal.pone.0319656
BibTeX
@article{Xie2025,
author = {Xie, Dongwei and Zhang, Xiaodan and Gao, Xiang and Zhao, Hu and Du, Dongyang},
title = {MAF-Net: A multimodal data fusion approach for human action recognition},
journal = {PLOS ONE},
year = {2025},
volume = {20},
number = {4},
pages = {e0319656},
doi = {10.1371/journal.pone.0319656}
}
EUVIMED