EUVIMEDEUVIMED entdecken
Crossref

Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning

Sunwoo Lee, Tuo Zhang, A. Salman Avestimehr

Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (7) · 8491-8499

DOI: 10.1609/aaai.v37i7.26023 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Sunwoo Lee und 2 weiteren Autor:innen. Die Arbeit wurde bisher 75-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In Federated Learning (FL), a common approach for aggregating local solutions across clients is periodic full model averaging. It is, however, known that different layers of neural networks can have a different degree of model discrepancy across the clients. The conventional full aggregation scheme does not consider such a difference and synchronizes the whole model parameters at once, resulting in inefficient network bandwidth consumption. Aggregating the parameters that are similar across the clients does not make meaningful training progress while increasing the communication cost. We propose FedLAMA, a layer-wise adaptive model aggregation scheme for scalable FL. FedLAMA adjusts the aggregation interval in a layer-wise manner, jointly considering the model discrepancy and the communication cost. This fine-grained aggregation strategy enables to reduce the communication cost without significantly harming the model accuracy. Our extensive empirical study shows that, as the aggregation interval increases, FedLAMA shows a remarkably smaller accuracy drop than the periodic full aggregation, while achieving comparable communication efficiency.

Zitieren

APA

Lee, S., Zhang, T., & Avestimehr, A. S. (2023). Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37(7), 8491-8499. https://doi.org/10.1609/aaai.v37i7.26023

Vancouver

Lee S, Zhang T, Avestimehr AS. Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(7):8491-8499. doi:10.1609/aaai.v37i7.26023

BibTeX
@article{Lee2023,
  author = {Lee, Sunwoo and Zhang, Tuo and Avestimehr, A. Salman},
  title = {Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2023},
  volume = {37},
  number = {7},
  pages = {8491--8499},
  doi = {10.1609/aaai.v37i7.26023}
}
Kostenlos starten →