FedMDS: An Efficient Model Discrepancy-Aware Semi-Asynchronous Clustered Federated Learning Framework
IEEE Transactions on Parallel and Distributed Systems · 01.03.2023 · 34 (3) · 1007-1019
DOI: 10.1109/tpds.2023.3237752 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Parallel and Distributed Systems, erschienen am 01.03.2023. Verfasst von Yu Zhang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 75-mal zitiert (Crossref) und nennt 41 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Chongqing Postdoctoral Science Foundation, Funds for Chongqing Distinguished Young Scholars.
Abstract (Originalsprache)
Federated learning (FL) is an emerging distributed machine learning paradigm that protects privacy and tackles the problem of isolated data islands. At present, there are two main communication strategies of FL: synchronous FL and asynchronous FL. The advantages of synchronous FL are the high precision and easy convergence of the model. However, this synchronous communication strategy has the risk of the straggler effect. Asynchronous FL has a natural advantage in mitigating the straggler effect, but there are threats of model quality degradation and server crash. In this paper, we propose a model discrepancy-aware semi-asynchronous clustered FL framework, FedMDS , which alleviates the straggler effect by 1) a clustered strategy based on the delay and direction of the model update and 2) a synchronous trigger mechanism that limits the model staleness. FedMDS leverages the clustered algorithm to reschedule the clients. Each group of clients performs asynchronous updates until the synchronous update mechanism based on the model discrepancy is triggered. We evaluate FedMDS based on four typical federated datasets in a non-IID setting and compare FedMDS to the baselines. The experimental results show that FedMDS significantly improves average test accuracy by more than $+9.2\%$ + 9 . 2 % on the four datasets compared to TA-FedAvg . In particular, FedMDS improves absolute Top-1 test accuracy by $+37.6\%$ + 37 . 6 % on FEMNIST compared to TA-FedAvg . The frequency of the average synchronization waiting time of FedMDS is significantly lower than that of TA-FedAvg on all datasets. Moreover, FedMDS can improve the accuracy and alleviate the straggler effect.
Förderung
- National Natural Science Foundation of China
- Chongqing Postdoctoral Science Foundation
- Funds for Chongqing Distinguished Young Scholars
Zitieren
APA
Zhang, Y., Liu, D., Duan, M., Li, L., Chen, X., Ren, A., Tan, Y., & Wang, C. (2023). FedMDS: An Efficient Model Discrepancy-Aware Semi-Asynchronous Clustered Federated Learning Framework. IEEE Transactions on Parallel and Distributed Systems, 34(3), 1007-1019. https://doi.org/10.1109/tpds.2023.3237752
Vancouver
Zhang Y, Liu D, Duan M, Li L, Chen X, Ren A, et al. FedMDS: An Efficient Model Discrepancy-Aware Semi-Asynchronous Clustered Federated Learning Framework. IEEE Transactions on Parallel and Distributed Systems. 2023;34(3):1007-1019. doi:10.1109/tpds.2023.3237752
BibTeX
@article{Zhang2023,
author = {Zhang, Yu and Liu, Duo and Duan, Moming and Li, Li and Chen, Xianzhang and Ren, Ao and Tan, Yujuan and Wang, Chengliang},
title = {FedMDS: An Efficient Model Discrepancy-Aware Semi-Asynchronous Clustered Federated Learning Framework},
journal = {IEEE Transactions on Parallel and Distributed Systems},
year = {2023},
volume = {34},
number = {3},
pages = {1007--1019},
doi = {10.1109/tpds.2023.3237752}
}
EUVIMED