EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation

Dan Lv, Zeyuan Wang

Electronics · 08.11.2026 · 15 (21) · 5187

DOI: 10.3390/electronics15215187 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 08.11.2026. Verfasst von Dan Lv und 1 weiteren Autor:innen. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Utilizing federated learning to fine-tune large language models on sensitive private data while achieving strong privacy protection without sacrificing model utility remains a fundamental challenge. Although parameter-efficient fine-tuning methods such as LoRA significantly reduce communication overhead, integrating them with differential privacy introduces issues including multiplicative noise amplification, cross-round noise accumulation, and consequent training instability, leading to severe performance degradation under strict privacy constraints. To address the above challenges, this paper proposes the Fed-PriLoRA (Private and Robust Federated Low-Rank Adaptation) framework. The framework decomposes each client’s low-rank update into separable vector pairs. It then employs structured freezing to suppress multiplicative amplification of differential privacy noise, and leverages Fisher information for noise-aware parameter importance evaluation to distinguish generalizable features from noise interference. By using Fisher information to assess parameter importance, the client locally discards low-contribution components to mitigate noise interference prior to uploading, thereby reducing communication overhead. Fed-PriLoRA achieves higher accuracy than the non-private StackLoRA baseline on both the Dolly-15K and WizardLM datasets. In terms of communication efficiency, the per-round communication overhead is reduced to just 0.051% of that incurred in full fine-tuning. Ablation experiments demonstrate that Fisher-guided pruning yields a relative accuracy increase of 39% over random pruning.

Förderung

  • Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China

Zitieren

APA

Lv, D., & Wang, Z. (2026). Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation. Electronics, 15(21), 5187. https://doi.org/10.3390/electronics15215187

Vancouver

Lv D, Wang Z. Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation. Electronics. 2026;15(21):5187. doi:10.3390/electronics15215187

BibTeX
@article{Lv2026,
  author = {Lv, Dan and Wang, Zeyuan},
  title = {Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {21},
  pages = {5187},
  doi = {10.3390/electronics15215187}
}
Kostenlos starten →