FLASH: Heterogeneity-Aware Federated Learning at Scale
IEEE Transactions on Mobile Computing · 2024-01 · 23 (1) · 483-500
DOI: 10.1109/tmc.2022.3214234 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2024-01. Verfasst von Chengxu Yang und 10 weiteren Autor:innen. Die Arbeit wurde bisher 43-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, National Natural Science Fund for the Excellent Young Scientists Fund Program.
Abstract (Originalsprache)
Federated learning (FL) becomes a promising machine learning paradigm. The impact of heterogeneous hardware specifications and dynamic states on the FL process has not yet been studied systematically. This paper presents the first large-scale study of this impact based on real-world data collected from 136k smartphones. We conducted extensive experiments on our proposed heterogeneity-aware FL platform namely FLASH, to systematically explore the performance of state-of-the-art FL algorithms and key FL configurations in heterogeneity-aware and -unaware settings, finding the following. (1) Heterogeneity causes accuracy to drop by up to 9.2% and convergence time to increase by 2.32×. (2) Heterogeneity negatively impacts popular aggregation algorithms, e.g., the accuracy variance reduction brought by q-FedAvg drops by 17.5%. (3) Heterogeneity does not worsen the accuracy loss caused by gradient-compression algorithms significantly, but it compromises the convergence time by up to 2.5×. (4) Heterogeneity hinders client-selection algorithms from selecting wanted clients, thus reducing effectiveness. e.g., the accuracy increase brought by the state-of-the-art client-selection algorithm drops by 73.9%. (5) Heterogeneity causes the optimal FL hyper-parameters to drift significantly. More specifically, the heterogeneity-unaware setting favors looser deadline and higher reporting fraction to achieve better training performance. (6) Heterogeneity results in non-trivial failed clients (more than 10%) and leads to participation bias (the top 30% of clients contribute 86% of computations). Our FLASH platform and data have been publicly open sourced.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- National Natural Science Fund for the Excellent Young Scientists Fund Program
- ERC Advanced Fellowship
Zitieren
APA
Yang, C., Xu, M., Wang, Q., Chen, Z., Huang, K., Ma, Y., Bian, K., Huang, G., Liu, Y., Jin, X., & Liu, X. (2024). FLASH: Heterogeneity-Aware Federated Learning at Scale. IEEE Transactions on Mobile Computing, 23(1), 483-500. https://doi.org/10.1109/tmc.2022.3214234
Vancouver
Yang C, Xu M, Wang Q, Chen Z, Huang K, Ma Y, et al. FLASH: Heterogeneity-Aware Federated Learning at Scale. IEEE Transactions on Mobile Computing. 2024;23(1):483-500. doi:10.1109/tmc.2022.3214234
BibTeX
@article{Yang2024,
author = {Yang, Chengxu and Xu, Mengwei and Wang, Qipeng and Chen, Zhenpeng and Huang, Kang and Ma, Yun and Bian, Kaigui and Huang, Gang and Liu, Yunxin and Jin, Xin and Liu, Xuanzhe},
title = {FLASH: Heterogeneity-Aware Federated Learning at Scale},
journal = {IEEE Transactions on Mobile Computing},
year = {2024},
volume = {23},
number = {1},
pages = {483--500},
doi = {10.1109/tmc.2022.3214234}
}
EUVIMED