Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation
Electronics · 08.11.2026 · 15 (21) · 5187
DOI: 10.3390/electronics15215187 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 08.11.2026. Verfasst von Dan Lv und 1 weiteren Autor:innen. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China.
Abstract (Originalsprache)
Utilizing federated learning to fine-tune large language models on sensitive private data while achieving strong privacy protection without sacrificing model utility remains a fundamental challenge. Although parameter-efficient fine-tuning methods such as LoRA significantly reduce communication overhead, integrating them with differential privacy introduces issues including multiplicative noise amplification, cross-round noise accumulation, and consequent training instability, leading to severe performance degradation under strict privacy constraints. To address the above challenges, this paper proposes the Fed-PriLoRA (Private and Robust Federated Low-Rank Adaptation) framework. The framework decomposes each client’s low-rank update into separable vector pairs. It then employs structured freezing to suppress multiplicative amplification of differential privacy noise, and leverages Fisher information for noise-aware parameter importance evaluation to distinguish generalizable features from noise interference. By using Fisher information to assess parameter importance, the client locally discards low-contribution components to mitigate noise interference prior to uploading, thereby reducing communication overhead. Fed-PriLoRA achieves higher accuracy than the non-private StackLoRA baseline on both the Dolly-15K and WizardLM datasets. In terms of communication efficiency, the per-round communication overhead is reduced to just 0.051% of that incurred in full fine-tuning. Ablation experiments demonstrate that Fisher-guided pruning yields a relative accuracy increase of 39% over random pruning.
Förderung
- Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China
Zitieren
APA
Lv, D., & Wang, Z. (2026). Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation. Electronics, 15(21), 5187. https://doi.org/10.3390/electronics15215187
Vancouver
Lv D, Wang Z. Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation. Electronics. 2026;15(21):5187. doi:10.3390/electronics15215187
BibTeX
@article{Lv2026,
author = {Lv, Dan and Wang, Zeyuan},
title = {Fed-PriLoRA: A Synergistic Framework for Private and Robust Federated Low-Rank Adaptation},
journal = {Electronics},
year = {2026},
volume = {15},
number = {21},
pages = {5187},
doi = {10.3390/electronics15215187}
}
EUVIMED