Federated two-stage decoupling with adaptive personalization layers
Complex & Intelligent Systems · 2024-06 · 10 (3) · 3657-3671
DOI: 10.1007/s40747-024-01342-1 ↗Kurz gefasst
Fachartikel in der Zeitschrift Complex & Intelligent Systems, online erschienen am 15.02.2024. Verfasst von Hangyu Zhu und 2 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Federated learning has gained significant attention due to its groundbreaking ability to enable distributed learning while maintaining privacy constraints. However, as a consequence of data heterogeneity among decentralized devices, it inherently experiences significant learning degradation and slow convergence speed. Therefore, it is natural to employ the concept of clustering homogeneous clients into the same group, allowing only the model weights within each group to be aggregated. While most existing clustered federated learning methods employ either model gradients or inference outputs as metrics for client partitioning to group similar devices together, heterogeneity may still exist within each cluster. Moreover, there is a scarcity of research exploring the underlying reasons for determining the appropriate timing for clustering, resulting in the common practice of assigning each client to its own individual cluster, particularly in the context of highly non-independent and identically distributed (Non-IID) data. In this paper, we introduce a two-stage decoupling federated learning algorithm with adaptive personalization layers named FedTSDP, where client clustering is performed twice according to inference outputs and model weights, respectively. Hopkins amended sampling is adopted to determine the appropriate timing for clustering and the sampling weight of public unlabeled data. In addition, a simple yet effective approach is developed to adaptively adjust the personalization layers based on varying degrees of data skew. Experimental results show that our proposed method has reliable performance on both IID and non-IID scenarios.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Zhu, H., Fan, Y., & Xie, Z. (2024). Federated two-stage decoupling with adaptive personalization layers. Complex & Intelligent Systems, 10(3), 3657-3671. https://doi.org/10.1007/s40747-024-01342-1
Vancouver
Zhu H, Fan Y, Xie Z. Federated two-stage decoupling with adaptive personalization layers. Complex & Intelligent Systems. 2024;10(3):3657-3671. doi:10.1007/s40747-024-01342-1
BibTeX
@article{Zhu2024,
author = {Zhu, Hangyu and Fan, Yuxiang and Xie, Zhenping},
title = {Federated two-stage decoupling with adaptive personalization layers},
journal = {Complex & Intelligent Systems},
year = {2024},
volume = {10},
number = {3},
pages = {3657--3671},
doi = {10.1007/s40747-024-01342-1}
}
EUVIMED