Communication-Efficient Distributed Learning: An Overview
IEEE Journal on Selected Areas in Communications · 2023-04 · 41 (4) · 851-873
DOI: 10.1109/jsac.2023.3242710 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Journal on Selected Areas in Communications, 2023-04. Verfasst von Xuanyu Cao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 142-mal zitiert (Crossref) und nennt 183 Literaturangaben. Gefördert von: U.S. National Science Foundation, U.S. Army Research Office under MURI, U.S. Army Research Laboratory Cooperative.
Abstract (Originalsprache)
Distributed learning is envisioned as the bedrock of next-generation intelligent networks, where intelligent agents, such as mobile devices, robots, and sensors, exchange information with each other or a parameter server to train machine learning models collaboratively without uploading raw data to a central entity for centralized processing. By utilizing the computation/communication capability of individual agents, the distributed learning paradigm can mitigate the burden at central processors and help preserve data privacy of users. Despite its promising applications, a downside of distributed learning is its need for iterative information exchange over wireless channels, which may lead to high communication overhead unaffordable in many practical systems with limited radio resources such as energy and bandwidth. To overcome this communication bottleneck, there is an urgent need for the development of communication-efficient distributed learning algorithms capable of reducing the communication cost and achieving satisfactory learning/optimization performance simultaneously. In this paper, we present a comprehensive survey of prevailing methodologies for communication-efficient distributed learning, including reduction of the number of communications, compression and quantization of the exchanged information, radio resource management for efficient learning, and game-theoretic mechanisms incentivizing user participation. We also point out potential directions for future research to further enhance the communication efficiency of distributed learning in various scenarios.
Förderung
- U.S. National Science Foundation
- U.S. Army Research Office under MURI
- U.S. Army Research Laboratory Cooperative
- National Natural Science Foundation of China
Zitieren
APA
Cao, X., Başar, T., Diggavi, S., Eldar, Y. C., Letaief, K. B., Poor, H. V., & Zhang, J. (2023). Communication-Efficient Distributed Learning: An Overview. IEEE Journal on Selected Areas in Communications, 41(4), 851-873. https://doi.org/10.1109/jsac.2023.3242710
Vancouver
Cao X, Başar T, Diggavi S, Eldar YC, Letaief KB, Poor HV, et al. Communication-Efficient Distributed Learning: An Overview. IEEE Journal on Selected Areas in Communications. 2023;41(4):851-873. doi:10.1109/jsac.2023.3242710
BibTeX
@article{Cao2023,
author = {Cao, Xuanyu and Başar, Tamer and Diggavi, Suhas and Eldar, Yonina C. and Letaief, Khaled B. and Poor, H. Vincent and Zhang, Junshan},
title = {Communication-Efficient Distributed Learning: An Overview},
journal = {IEEE Journal on Selected Areas in Communications},
year = {2023},
volume = {41},
number = {4},
pages = {851--873},
doi = {10.1109/jsac.2023.3242710}
}
EUVIMED