A Unified AI-Driven Multimodal Framework Integrating Visual Sensing and Wearable Sensors for Robust Human Motion Monitoring in Biomedical Applications
Sensors · 2026
DOI: 10.3390/s26082314 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2026. Verfasst von Qiang Chen und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Physical Education and Military Affairs, China Jiliang University, Hangzhou 310018, China. Hauptthemen laut MEDLINE: Wearable Electronic Devices, Artificial Intelligence, Biosensing Techniques. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
This study proposes a unified multimodal temporal motion state perception framework for optical imaging-oriented biomedical applications, integrating visual skeleton sequences, inertial measurement unit (IMU) signals, and surface electromyography (EMG) signals. The framework utilizes modality-specific encoders and a cross-modal temporal alignment attention mechanism to explicitly model temporal offsets from heterogeneous sensing streams. A multimodal temporal Transformer backbone is introduced to capture long-range motion dependencies and cross-modal interactions, while an uncertainty-aware fusion module dynamically allocates weights based on modality confidence. Experimental results demonstrate that the proposed approach achieves an accuracy of 94.37%, an F1-score of 93.95%, and a mean average precision of 96.02%, outperforming mainstream baseline models. Robustness evaluations further confirm stable performance under visual occlusion and sensor noise. These results indicate that the framework provides a highly accurate and robust solution for rehabilitation assessment, sports training monitoring, and wearable intelligent interaction systems.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Physical Education and Military Affairs, China Jiliang University, Hangzhou 310018, China.
- National School of Development, Peking University, Beijing 100871, China.
- China Agricultural University, Beijing 100083, China.
- Artificial Intelligence Research Institute, Tsinghua University, Beijing 100084, China.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Chen, Q., Wang, X., Chen, R., Hua, S., Li, Y., Liu, S., & Zhan, Y. (2026). A Unified AI-Driven Multimodal Framework Integrating Visual Sensing and Wearable Sensors for Robust Human Motion Monitoring in Biomedical Applications. Sensors. https://doi.org/10.3390/s26082314
Vancouver
Chen Q, Wang X, Chen R, Hua S, Li Y, Liu S, et al. A Unified AI-Driven Multimodal Framework Integrating Visual Sensing and Wearable Sensors for Robust Human Motion Monitoring in Biomedical Applications. Sensors. 2026. doi:10.3390/s26082314
BibTeX
@article{Chen2026,
author = {Chen, Qiang and Wang, Xiaoya and Chen, Ranran and Hua, Surui and Li, Yufei and Liu, Siyuan and Zhan, Yan},
title = {A Unified AI-Driven Multimodal Framework Integrating Visual Sensing and Wearable Sensors for Robust Human Motion Monitoring in Biomedical Applications},
journal = {Sensors},
year = {2026},
doi = {10.3390/s26082314}
}
EUVIMED