Exploring Representativity in Device Scheduling for Wireless Federated Learning
IEEE Transactions on Wireless Communications · 2024-01 · 23 (1) · 720-735
DOI: 10.1109/twc.2023.3281765 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Wireless Communications, 2024-01. Verfasst von Zhixiong Chen und 2 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 41 Literaturangaben. Gefördert von: Engineering and Physical Sciences Research Council (EPSRC), U.K..
Abstract (Originalsprache)
Existing device scheduling works in wireless federated learning (FL) mainly focused on selecting the devices with maximum gradient norm or loss function and require all devices to perform local training in each round. This may produce extra training costs and schedule devices with similar data statistics, thus degrading learning performance. To mitigate these problems, we first theoretically characterize the convergence behaviour of the considered FL system, finding that the learning performance is degraded by the difference between the aggregated gradient of scheduled devices and the full participation gradient. Inspired by this, we propose to find a subset of representative devices and the corresponding pre-device stepsizes to approximate the full participation aggregated gradient. Considering the limited wireless bandwidth, we formulate a problem to capture the trade-off between representativity and latency by optimizing device scheduling and bandwidth allocation policies. Our analysis reveals optimal bandwidth allocation is achieved when all scheduled devices have the same latency. Then, by proving the non-monotone submodularity of the problem, we develop a double greedy algorithm to solve the device scheduling policy. To avoid the local training of unscheduled devices, we utilize the historical gradient information of devices to estimate the current gradient for device scheduling design. Compared to existing scheduling algorithms, the proposed representativity-aware device scheduling algorithm improves 6.7% and 4.02% accuracies on two typical datasets under heterogeneous local data distributions, i.e., MNIST and CIFAR-10, respectively. In addition, the proposed latency- and representativity-aware scheduling algorithm saves over 16% and 12% training time for MNIST and CIFAR-10 datasets than the scheduling algorithms based on either latency and representativity individually.
Förderung
- Engineering and Physical Sciences Research Council (EPSRC), U.K.
Zitieren
APA
Chen, Z., Yi, W., & Nallanathan, A. (2024). Exploring Representativity in Device Scheduling for Wireless Federated Learning. IEEE Transactions on Wireless Communications, 23(1), 720-735. https://doi.org/10.1109/twc.2023.3281765
Vancouver
Chen Z, Yi W, Nallanathan A. Exploring Representativity in Device Scheduling for Wireless Federated Learning. IEEE Transactions on Wireless Communications. 2024;23(1):720-735. doi:10.1109/twc.2023.3281765
BibTeX
@article{Chen2024,
author = {Chen, Zhixiong and Yi, Wenqiang and Nallanathan, Arumugam},
title = {Exploring Representativity in Device Scheduling for Wireless Federated Learning},
journal = {IEEE Transactions on Wireless Communications},
year = {2024},
volume = {23},
number = {1},
pages = {720--735},
doi = {10.1109/twc.2023.3281765}
}
EUVIMED