FedMGL: Boosting One-Shot Federated Learning via Mixture of Grouped LoRA Experts
IEEE Transactions on Circuits and Systems for Video Technology · 2026-11 · 36 (11) · 16805-16819
DOI: 10.1109/tcsvt.2026.3714461 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-11. Verfasst von Ke Xu und 5 weiteren Autor:innen. Die Arbeit nennt 94 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Federated tuning methods typically involve full parameter fine-tuning and require multiple rounds of communication with the server. However, the communication budget resulting from fully fine-tuning is impractical in the real world. Combining the One-Shot FL paradigm and parameter-efficient fine-tuning methods, such as LoRA, can reduce the communication budget hundreds of times, but model aggregation performance will be seriously degraded and lacks robustness. Current OFL is typically based on full-parameter training, which hinders the practical application of OFL. Based on the LoRA trained via OFL, we propose a method by boosting One-Shot Federated tuning with Mixture of Grouped LoRAs called FedMGL that significantly improves performance without training the LoRA parameters themselves. To reduce communication and training overhead, FedMGL first performs similarity grouping and aggregation for the locally trained LoRA matrix. To achieve learnable aggregation, we freeze the grouped LoRA modules and locally fine-tune gating units on the client or server side, requiring minimal training overhead. Additionally, it learns better aggregation ratios, making the final aggregated parameters more robust. Extensive experiments show that FedMGL has superior performance on various settings without compromising data privacy or incurring large communication costs.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Xu, K., Wang, Y., Zhang, Y., Xiao, B., Chen, C., & Zhang, X. (2026). FedMGL: Boosting One-Shot Federated Learning via Mixture of Grouped LoRA Experts. IEEE Transactions on Circuits and Systems for Video Technology, 36(11), 16805-16819. https://doi.org/10.1109/tcsvt.2026.3714461
Vancouver
Xu K, Wang Y, Zhang Y, Xiao B, Chen C, Zhang X. FedMGL: Boosting One-Shot Federated Learning via Mixture of Grouped LoRA Experts. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(11):16805-16819. doi:10.1109/tcsvt.2026.3714461
BibTeX
@article{Xu2026,
author = {Xu, Ke and Wang, Yixin and Zhang, Yeming and Xiao, Bo and Chen, Cuiqun and Zhang, Xingyi},
title = {FedMGL: Boosting One-Shot Federated Learning via Mixture of Grouped LoRA Experts},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {11},
pages = {16805--16819},
doi = {10.1109/tcsvt.2026.3714461}
}
EUVIMED