Communication-Efficient Design for Quantized Decentralized Federated Learning
IEEE Transactions on Signal Processing · 2024 · 72 · 1175-1188
DOI: 10.1109/tsp.2024.3363887 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Signal Processing, 2024. Verfasst von Li Chen und 3 weiteren Autor:innen. Die Arbeit wurde bisher 57-mal zitiert (Crossref) und nennt 38 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Anhui Province.
Abstract (Originalsprache)
Decentralized federated learning (DFL) is a variant of federated learning, where edge nodes only communicate with their one-hop neighbors to learn the optimal model. However, as information exchange is restricted in a range of one-hop in DFL, inefficient information exchange leads to more communication rounds to reach the targeted training loss. This greatly reduces the communication efficiency. In this paper, we propose a new non-uniform quantization of model parameters to improve DFL convergence. Specifically, we apply the Lloyd-Max algorithm to DFL (LM-DFL) first to minimize the quantization distortion by adjusting the quantization levels adaptively. Convergence guarantee of LM-DFL is established without convex loss assumption. Based on LM-DFL, we then propose a new doubly-adaptive DFL, which jointly considers the ascending number of quantization levels to reduce the amount of communicated information in the training and adapts the quantization levels for non-uniform gradient distributions. Experiment results based on MNIST and CIFAR-10 datasets illustrate the superiority of LM-DFL with the optimal quantized distortion and show that doubly-adaptive DFL can greatly improve communication efficiency.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Anhui Province
Zitieren
APA
Chen, L., Liu, W., Chen, Y., & Wang, W. (2024). Communication-Efficient Design for Quantized Decentralized Federated Learning. IEEE Transactions on Signal Processing, 72, 1175-1188. https://doi.org/10.1109/tsp.2024.3363887
Vancouver
Chen L, Liu W, Chen Y, Wang W. Communication-Efficient Design for Quantized Decentralized Federated Learning. IEEE Transactions on Signal Processing. 2024;72:1175-1188. doi:10.1109/tsp.2024.3363887
BibTeX
@article{Chen2024,
author = {Chen, Li and Liu, Wei and Chen, Yunfei and Wang, Weidong},
title = {Communication-Efficient Design for Quantized Decentralized Federated Learning},
journal = {IEEE Transactions on Signal Processing},
year = {2024},
volume = {72},
pages = {1175--1188},
doi = {10.1109/tsp.2024.3363887}
}
EUVIMED