FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (15) · 16768-16776
DOI: 10.1609/aaai.v38i15.29617 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Jianqing Zhang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 140-mal zitiert (Crossref).
Abstract (Originalsprache)
Recently, Heterogeneous Federated Learning (HtFL) has attracted attention due to its ability to support heterogeneous models and data. To reduce the high communication cost of transmitting model parameters, a major challenge in HtFL, prototype-based HtFL methods are proposed to solely share class representatives, a.k.a, prototypes, among heterogeneous clients while maintaining the privacy of clients’ models. However, these prototypes are naively aggregated into global prototypes on the server using weighted averaging, resulting in suboptimal global knowledge which negatively impacts the performance of clients. To overcome this challenge, we introduce a novel HtFL approach called FedTGP, which leverages our Adaptive-margin-enhanced Contrastive Learning (ACL) to learn Trainable Global Prototypes (TGP) on the server. By incorporating ACL, our approach enhances prototype separability while preserving semantic meaning. Extensive experiments with twelve heterogeneous models demonstrate that our FedTGP surpasses state-of-the-art methods by up to 9.08% in accuracy while maintaining the communication and privacy advantages of prototype-based HtFL. Our code is available at https://github.com/TsingZ0/FedTGP.
Zitieren
APA
Zhang, J., Liu, Y., Hua, Y., & Cao, J. (2024). FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15), 16768-16776. https://doi.org/10.1609/aaai.v38i15.29617
Vancouver
Zhang J, Liu Y, Hua Y, Cao J. FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(15):16768-16776. doi:10.1609/aaai.v38i15.29617
BibTeX
@article{Zhang2024,
author = {Zhang, Jianqing and Liu, Yang and Hua, Yang and Cao, Jian},
title = {FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {15},
pages = {16768--16776},
doi = {10.1609/aaai.v38i15.29617}
}
EUVIMED