Fed-OGD: Mitigating Straggler Effects in Federated Learning via Orthogonal Gradient Descent
IEEE Transactions on Computers · 2025-09 · 74 (9) · 3018-3031
DOI: 10.1109/tc.2025.3584272 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Computers, 2025-09. Verfasst von Wei Li und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Lingnan University.
Abstract (Originalsprache)
Federated Learning (FL) faces challenges due to straggler clients that impede timely parameter uploads, potentially leading to suboptimal global model performance. Existing approaches using synchronous and asynchronous communication suffer from long waiting times or convergence issues. We propose Fed-OGD, a novel asynchronous FL method addressing the straggler problem through gradient orthogonalization. Our approach innovatively frames the straggler issue using catastrophic forgetting theory, viewing stragglers as instances of the global model “forgetting” to aggregate their parameters. Fed-OGD introduces an Orthogonal Gradient Descent (OGD) technique that caches straggler gradients and orthogonalizes the difference between these and current active client gradients. By projecting active gradients onto straggler orthogonal bases and subtracting the resulting components, we obtain orthogonalized gradients guiding the model towards optimality. We provide theoretical convergence guarantees and demonstrate Fed-OGD’s effectiveness through extensive experiments. Our method achieves state-of-the-art performance across multiple datasets among SOTA FL baselines, with notable improvements in non-IID (non-Independent and identically distributed) scenarios: there are few main categories with many samples while other categories hold few samples in a client. Fed-OGD achieves that 16.66% increase in accuracy on CIFAR-10, and significant gains on CIFAR-100 (5.37%), Tiny-ImageNet (38.51%), and AG_NEWS (16.30%).
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Lingnan University
- Lam Woo Research Fund at LU
Zitieren
APA
Li, W., Shen, Z., Liu, X., Ding, C., & Shen, J. (2025). Fed-OGD: Mitigating Straggler Effects in Federated Learning via Orthogonal Gradient Descent. IEEE Transactions on Computers, 74(9), 3018-3031. https://doi.org/10.1109/tc.2025.3584272
Vancouver
Li W, Shen Z, Liu X, Ding C, Shen J. Fed-OGD: Mitigating Straggler Effects in Federated Learning via Orthogonal Gradient Descent. IEEE Transactions on Computers. 2025;74(9):3018-3031. doi:10.1109/tc.2025.3584272
BibTeX
@article{Li2025,
author = {Li, Wei and Shen, Zicheng and Liu, Xiulong and Ding, Chuntao and Shen, Jiaxing},
title = {Fed-OGD: Mitigating Straggler Effects in Federated Learning via Orthogonal Gradient Descent},
journal = {IEEE Transactions on Computers},
year = {2025},
volume = {74},
number = {9},
pages = {3018--3031},
doi = {10.1109/tc.2025.3584272}
}
EUVIMED