Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Training
IEEE Transactions on Mobile Computing · 2024-12 · 23 (12) · 11133-11143
DOI: 10.1109/tmc.2024.3392212 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2024-12. Verfasst von Sunwoo Lee und 4 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 49 Literaturangaben. Gefördert von: ARO, ONR, Institute of Information and Communications Technology Planning & Evaluation.
Abstract (Originalsprache)
In Federated Learning (FL), clients may have weak devices that cannot train the full model or even hold it in their memory space. To implement large-scale FL applications, thus, it is crucial to develop a distributed learning method that enables the participation of such weak clients. We propose $\mathtt{EmbracingFL}$ EmbracingFL , a general FL framework that allows all available clients to join the distributed training regardless of their system resource capacity. The framework is built upon a novel form of partial model training method in which each client trains as many consecutive output-side layers as its system resources allow. Our study demonstrates that $\mathtt{EmbracingFL}$ EmbracingFL encourages each layer to have similar data representations across clients, improving FL efficiency. The proposed partial model training method guarantees convergence to a neighbor of stationary points for non-convex and smooth problems. We evaluate the efficacy of $\mathtt{EmbracingFL}$ EmbracingFL under a variety of settings with a mixed number of strong, moderate ( $\sim\! 40\%$ ∼ 40 % memory), and weak ( $\sim\! 15\%$ ∼ 15 % memory) clients, datasets (CIFAR-10, FEMNIST, and IMDB), and models (ResNet20, CNN, and LSTM). Our empirical study shows that $\mathtt{EmbracingFL}$ EmbracingFL consistently achieves high accuracy as like all clients are strong, outperforming the state-of-the-art width reduction methods (i.e., HeteroFL and FjORD).
Förderung
- ARO
- ONR
- Institute of Information and Communications Technology Planning & Evaluation
- Inha University
- National Research Foundation of Korea
- INHA UNIVERSITY Research
Zitieren
APA
Lee, S., Zhang, T., Prakash, S., Niu, Y., & Avestimehr, S. (2024). Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Training. IEEE Transactions on Mobile Computing, 23(12), 11133-11143. https://doi.org/10.1109/tmc.2024.3392212
Vancouver
Lee S, Zhang T, Prakash S, Niu Y, Avestimehr S. Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Training. IEEE Transactions on Mobile Computing. 2024;23(12):11133-11143. doi:10.1109/tmc.2024.3392212
BibTeX
@article{Lee2024,
author = {Lee, Sunwoo and Zhang, Tuo and Prakash, Saurav and Niu, Yue and Avestimehr, Salman},
title = {Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Training},
journal = {IEEE Transactions on Mobile Computing},
year = {2024},
volume = {23},
number = {12},
pages = {11133--11143},
doi = {10.1109/tmc.2024.3392212}
}
EUVIMED