EUVIMEDEUVIMED entdecken
Crossref

FedMGL: Boosting One-Shot Federated Learning via Mixture of Grouped LoRA Experts

Ke Xu, Yixin Wang, Yeming Zhang, Bo Xiao, Cuiqun Chen, Xingyi Zhang

IEEE Transactions on Circuits and Systems for Video Technology · 2026-11 · 36 (11) · 16805-16819

DOI: 10.1109/tcsvt.2026.3714461 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-11. Verfasst von Ke Xu und 5 weiteren Autor:innen. Die Arbeit nennt 94 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Federated tuning methods typically involve full parameter fine-tuning and require multiple rounds of communication with the server. However, the communication budget resulting from fully fine-tuning is impractical in the real world. Combining the One-Shot FL paradigm and parameter-efficient fine-tuning methods, such as LoRA, can reduce the communication budget hundreds of times, but model aggregation performance will be seriously degraded and lacks robustness. Current OFL is typically based on full-parameter training, which hinders the practical application of OFL. Based on the LoRA trained via OFL, we propose a method by boosting One-Shot Federated tuning with Mixture of Grouped LoRAs called FedMGL that significantly improves performance without training the LoRA parameters themselves. To reduce communication and training overhead, FedMGL first performs similarity grouping and aggregation for the locally trained LoRA matrix. To achieve learnable aggregation, we freeze the grouped LoRA modules and locally fine-tune gating units on the client or server side, requiring minimal training overhead. Additionally, it learns better aggregation ratios, making the final aggregated parameters more robust. Extensive experiments show that FedMGL has superior performance on various settings without compromising data privacy or incurring large communication costs.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Xu, K., Wang, Y., Zhang, Y., Xiao, B., Chen, C., & Zhang, X. (2026). FedMGL: Boosting One-Shot Federated Learning via Mixture of Grouped LoRA Experts. IEEE Transactions on Circuits and Systems for Video Technology, 36(11), 16805-16819. https://doi.org/10.1109/tcsvt.2026.3714461

Vancouver

Xu K, Wang Y, Zhang Y, Xiao B, Chen C, Zhang X. FedMGL: Boosting One-Shot Federated Learning via Mixture of Grouped LoRA Experts. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(11):16805-16819. doi:10.1109/tcsvt.2026.3714461

BibTeX
@article{Xu2026,
  author = {Xu, Ke and Wang, Yixin and Zhang, Yeming and Xiao, Bo and Chen, Cuiqun and Zhang, Xingyi},
  title = {FedMGL: Boosting One-Shot Federated Learning via Mixture of Grouped LoRA Experts},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2026},
  volume = {36},
  number = {11},
  pages = {16805--16819},
  doi = {10.1109/tcsvt.2026.3714461}
}
Kostenlos starten →