EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation

Dan Lv, Zeyuan Wang

Electronics · 2026-11-08 · 15 (21) · 5187

DOI: 10.3390/electronics15215187 ↗

At a glance

Journal article in Electronics, published online on 2026-11-08. Written by Dan Lv and 1 co-authors. 21 references. The full text is freely available (open access, CC BY 4.0). Funded by: Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China.

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Utilizing federated learning to fine-tune large language models on sensitive private data while achieving strong privacy protection without sacrificing model utility remains a fundamental challenge. Although parameter-efficient fine-tuning methods such as LoRA significantly reduce communication overhead, integrating them with differential privacy introduces issues including multiplicative noise amplification, cross-round noise accumulation, and consequent training instability, leading to severe performance degradation under strict privacy constraints. To address the above challenges, this paper proposes the Fed-PriLoRA (Private and Robust Federated Low-Rank Adaptation) framework. The framework decomposes each client’s low-rank update into separable vector pairs. It then employs structured freezing to suppress multiplicative amplification of differential privacy noise, and leverages Fisher information for noise-aware parameter importance evaluation to distinguish generalizable features from noise interference. By using Fisher information to assess parameter importance, the client locally discards low-contribution components to mitigate noise interference prior to uploading, thereby reducing communication overhead. Fed-PriLoRA achieves higher accuracy than the non-private StackLoRA baseline on both the Dolly-15K and WizardLM datasets. In terms of communication efficiency, the per-round communication overhead is reduced to just 0.051% of that incurred in full fine-tuning. Ablation experiments demonstrate that Fisher-guided pruning yields a relative accuracy increase of 39% over random pruning.

Funding

  • Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China

Cite

APA

Lv, D., & Wang, Z. (2026). Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation. Electronics, 15(21), 5187. https://doi.org/10.3390/electronics15215187

Vancouver

Lv D, Wang Z. Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation. Electronics. 2026;15(21):5187. doi:10.3390/electronics15215187

BibTeX
@article{Lv2026,
  author = {Lv, Dan and Wang, Zeyuan},
  title = {Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {21},
  pages = {5187},
  doi = {10.3390/electronics15215187}
}
Start for free →