Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning
Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (7) · 8491-8499
DOI: 10.1609/aaai.v37i7.26023 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Sunwoo Lee und 2 weiteren Autor:innen. Die Arbeit wurde bisher 75-mal zitiert (Crossref).
Abstract (Originalsprache)
In Federated Learning (FL), a common approach for aggregating local solutions across clients is periodic full model averaging. It is, however, known that different layers of neural networks can have a different degree of model discrepancy across the clients. The conventional full aggregation scheme does not consider such a difference and synchronizes the whole model parameters at once, resulting in inefficient network bandwidth consumption. Aggregating the parameters that are similar across the clients does not make meaningful training progress while increasing the communication cost. We propose FedLAMA, a layer-wise adaptive model aggregation scheme for scalable FL. FedLAMA adjusts the aggregation interval in a layer-wise manner, jointly considering the model discrepancy and the communication cost. This fine-grained aggregation strategy enables to reduce the communication cost without significantly harming the model accuracy. Our extensive empirical study shows that, as the aggregation interval increases, FedLAMA shows a remarkably smaller accuracy drop than the periodic full aggregation, while achieving comparable communication efficiency.
Zitieren
APA
Lee, S., Zhang, T., & Avestimehr, A. S. (2023). Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37(7), 8491-8499. https://doi.org/10.1609/aaai.v37i7.26023
Vancouver
Lee S, Zhang T, Avestimehr AS. Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(7):8491-8499. doi:10.1609/aaai.v37i7.26023
BibTeX
@article{Lee2023,
author = {Lee, Sunwoo and Zhang, Tuo and Avestimehr, A. Salman},
title = {Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2023},
volume = {37},
number = {7},
pages = {8491--8499},
doi = {10.1609/aaai.v37i7.26023}
}
EUVIMED