Multi-Level Personalized Federated Learning on Heterogeneous and Long-Tailed Data
IEEE Transactions on Mobile Computing · 2024-12 · 23 (12) · 12396-12409
DOI: 10.1109/tmc.2024.3409159 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2024-12. Verfasst von Rongyu Zhang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 41-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Hetao Shenzhen-HK S&T Cooperation Zone, Shenzhen Science and Technology Program.
Abstract (Originalsprache)
Federated learning (FL) offers a privacy-centric distributed learning framework, enabling model training on individual clients and central aggregation without necessitating data exchange. Nonetheless, FL implementations often suffer from non-i.i.d. and long-tailed class distributions across mobile applications, e.g., autonomous vehicles, which leads models to overfitting as local training may converge to sub-optimal. In our study, we explore the impact of data heterogeneity on model bias and introduce an innovative personalized FL framework, Multi-level Personalized Federated Learning (MuPFL), which leverages the hierarchical architecture of FL to fully harness computational resources at various levels. This framework integrates three pivotal modules: Biased Activation Value Dropout (BAVD) to mitigate overfitting and accelerate training; Adaptive Cluster-based Model Update (ACMU) to refine local models ensuring coherent global aggregation; and Prior Knowledge-assisted Classifier Fine-tuning (PKCF) to bolster classification and personalize models in accord with skewed local data with shared knowledge. Extensive experiments on diverse real-world datasets for image classification and semantic segmentation validate that MuPFL consistently outperforms state-of-the-art baselines, even under extreme non-i.i.d. and long-tail conditions, which enhances accuracy by as much as 7.39% and accelerates training by up to 80% at most, marking significant advancements in both efficiency and effectiveness.
Förderung
- National Natural Science Foundation of China
- Hetao Shenzhen-HK S&T Cooperation Zone
- Shenzhen Science and Technology Program
- Basic and Applied Basic Research Foundation of Guangdong Province
- Shenzhen Outstanding Talents Training Fund
- Guangdong Research Projects
- Guangdong Provincial Key Laboratory of Future Networks of Intelligence
- Shenzhen Key Laboratory of Big Data and Artificial Intelligence
Zitieren
APA
Zhang, R., Chen, Y., Wu, C., Wang, F., & Li, B. (2024). Multi-Level Personalized Federated Learning on Heterogeneous and Long-Tailed Data. IEEE Transactions on Mobile Computing, 23(12), 12396-12409. https://doi.org/10.1109/tmc.2024.3409159
Vancouver
Zhang R, Chen Y, Wu C, Wang F, Li B. Multi-Level Personalized Federated Learning on Heterogeneous and Long-Tailed Data. IEEE Transactions on Mobile Computing. 2024;23(12):12396-12409. doi:10.1109/tmc.2024.3409159
BibTeX
@article{Zhang2024,
author = {Zhang, Rongyu and Chen, Yun and Wu, Chenrui and Wang, Fangxin and Li, Bo},
title = {Multi-Level Personalized Federated Learning on Heterogeneous and Long-Tailed Data},
journal = {IEEE Transactions on Mobile Computing},
year = {2024},
volume = {23},
number = {12},
pages = {12396--12409},
doi = {10.1109/tmc.2024.3409159}
}
EUVIMED