EUVIMEDEUVIMED entdecken
Crossref

Cross-Modal Federated Human Activity Recognition

Xiaoshan Yang, Baochen Xiong, Yi Huang, Changsheng Xu

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-08 · 46 (8) · 5345-5361

DOI: 10.1109/tpami.2024.3367412 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-08. Verfasst von Xiaoshan Yang und 3 weiteren Autor:innen. Hauptthemen laut MEDLINE: Human Activities, Pattern Recognition, Automated, Algorithms. Die Arbeit wurde bisher 50-mal zitiert (Crossref) und nennt 86 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Federated human activity recognition (FHAR) has attracted much attention due to its great potential in privacy protection. Existing FHAR methods can collaboratively learn a global activity recognition model based on unimodal or multimodal data distributed on different local clients. However, it is still questionable whether existing methods can work well in a more common scenario where local data are from different modalities, e.g., some local clients may provide motion signals while others can only provide visual data. In this article, we study a new problem of cross-modal federated human activity recognition (CM-FHAR), which is conducive to promote the large-scale use of the HAR model on more local devices. CM-FHAR has at least three dedicated challenges: 1) distributive common cross-modal feature learning, 2) modality-dependent discriminate feature learning, 3) modality imbalance issue. To address these challenges, we propose a modality-collaborative activity recognition network (MCARN), which can comprehensively learn a global activity classifier shared across all clients and multiple modality-dependent private activity classifiers. To produce modality-agnostic and modality-specific features, we learn an altruistic encoder and an egocentric encoder under the constraint of a separation loss and an adversarial modality discriminator collaboratively learned in hyper-sphere. To address the modality imbalance issue, we propose an angular margin adjustment scheme to improve the modality discriminator on modality-imbalanced data by enhancing the intra-modality compactness of the dominant modality and increase the inter-modality discrepancy. Moreover, we propose a relation-aware global-local calibration mechanism to constrain class-level pairwise relationships for the parameters of the private classifier. Finally, through decentralized optimization with alternative steps of adversarial local updating and modality-aware global aggregation, the proposed MCARN obtains state-of-the-art performance on both modality-balanced and modality-imbalanced data.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Human ActivitiesPattern Recognition, AutomatedAlgorithmsHumansMachine Learning

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Yang, X., Xiong, B., Huang, Y., & Xu, C. (2024). Cross-Modal Federated Human Activity Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8), 5345-5361. https://doi.org/10.1109/tpami.2024.3367412

Vancouver

Yang X, Xiong B, Huang Y, Xu C. Cross-Modal Federated Human Activity Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(8):5345-5361. doi:10.1109/tpami.2024.3367412

BibTeX
@article{Yang2024,
  author = {Yang, Xiaoshan and Xiong, Baochen and Huang, Yi and Xu, Changsheng},
  title = {Cross-Modal Federated Human Activity Recognition},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {8},
  pages = {5345--5361},
  doi = {10.1109/tpami.2024.3367412}
}
Kostenlos starten →