Faster Adaptive Federated Learning
Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (9) · 10379-10387
DOI: 10.1609/aaai.v37i9.26235 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Xidong Wu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 79-mal zitiert (Crossref).
Abstract (Originalsprache)
Federated learning has attracted increasing attention with the emergence of distributed data. While extensive federated learning algorithms have been proposed for the non-convex distributed problem, the federated learning in practice still faces numerous challenges, such as the large training iterations to converge since the sizes of models and datasets keep increasing, and the lack of adaptivity by SGD-based model updates. Meanwhile, the study of adaptive methods in federated learning is scarce and existing works either lack a complete theoretical convergence guarantee or have slow sample complexity. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on the momentum-based variance reduced technique in cross-silo FL. We first explore how to design the adaptive algorithm in the FL setting. By providing a counter-example, we prove that a simple combination of FL and adaptive methods could lead to divergence. More importantly, we provide a convergence analysis for our method and prove that our algorithm is the first adaptive FL algorithm to reach the best-known samples O(epsilon(-3)) and O(epsilon(-2)) communication rounds to find an epsilon-stationary point without large batches. The experimental results on the language modeling task and image classification task with heterogeneous data demonstrate the efficiency of our algorithms.
Zitieren
APA
Wu, X., Huang, F., Hu, Z., & Huang, H. (2023). Faster Adaptive Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37(9), 10379-10387. https://doi.org/10.1609/aaai.v37i9.26235
Vancouver
Wu X, Huang F, Hu Z, Huang H. Faster Adaptive Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(9):10379-10387. doi:10.1609/aaai.v37i9.26235
BibTeX
@article{Wu2023,
author = {Wu, Xidong and Huang, Feihu and Hu, Zhengmian and Huang, Heng},
title = {Faster Adaptive Federated Learning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2023},
volume = {37},
number = {9},
pages = {10379--10387},
doi = {10.1609/aaai.v37i9.26235}
}
EUVIMED