EUVIMEDEUVIMED entdecken
Crossref

MASTER: A Multi-modal Foundation Model for Human Activity Recognition

Guanzhou Zhu, Dong Zhao, Chunliang Li, Mingyue Zhao, Zhengyuan Zhang, Hefeng Quan, Huadong Ma

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · 03.09.2025 · 9 (3) · 1-26

DOI: 10.1145/3749511 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, online erschienen am 03.09.2025. Verfasst von Guanzhou Zhu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 61 Literaturangaben. Gefördert von: National Major Scientific Instruments and Equipments Development Project of National Natural Science Foundation of China, National Natural Science Foundation of China, Beijing Natural Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-modal sensing has become crucial in Human Activity Recognition (HAR) due to its ability to combine data from diverse sensors. However, challenges arise in recognizing various activities in different scenes using multi-modal data from different positions and devices, due to dynamic combinations of modal inputs, data heterogeneity, and scarcity of labeled data. To tackle these challenges, we propose MASTER, a multi-modal foundation model specifically designed for HAR. MASTER introduces a masked-data modeling-based self-supervised pre-training method, enabling the model to learn from unlabeled data and adapt to dynamic combinations of modal inputs. Moreover, it incorporates a few-shot alignment mechanism to facilitate adaptation to different activities, scenes, positions, and devices. Through the pre-training and fine-tuning on 7 multi-modal HAR datasets, MASTER currently supports, but is not limited to, 8 modalities (ACC, Gyro, mmWave, WiFi, Skeleton, Lidar, Infrared, and RGB) and 45 human activities. The results demonstrate that MASTER achieves the highest accuracy with minimal labeled data across various situations, surpassing alternative solutions.

Förderung

  • National Major Scientific Instruments and Equipments Development Project of National Natural Science Foundation of China
  • National Natural Science Foundation of China
  • Beijing Natural Science Foundation
  • 111 Project
  • BUPT Excellent Ph.D. Students Foundation

Zitieren

APA

Zhu, G., Zhao, D., Li, C., Zhao, M., Zhang, Z., Quan, H., & Ma, H. (2025). MASTER: A Multi-modal Foundation Model for Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 9(3), 1-26. https://doi.org/10.1145/3749511

Vancouver

Zhu G, Zhao D, Li C, Zhao M, Zhang Z, Quan H, et al. MASTER: A Multi-modal Foundation Model for Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2025;9(3):1-26. doi:10.1145/3749511

BibTeX
@article{Zhu2025,
  author = {Zhu, Guanzhou and Zhao, Dong and Li, Chunliang and Zhao, Mingyue and Zhang, Zhengyuan and Quan, Hefeng and Ma, Huadong},
  title = {MASTER: A Multi-modal Foundation Model for Human Activity Recognition},
  journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
  year = {2025},
  volume = {9},
  number = {3},
  pages = {1--26},
  doi = {10.1145/3749511}
}
Kostenlos starten →