A Communication-Efficient, Privacy-Preserving Federated Learning Algorithm Based on Two-Stage Gradient Pruning and Differentiated Differential Privacy
Sensors · 21.11.2023 · 23 (23) · 9305
DOI: 10.3390/s23239305 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 21.11.2023. Verfasst von Yong Li und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Engineering, Changchun University of Technology, Changchun 130012, China. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Science and Technology Research Planning Project of Jilin Provincial Department of Education in China.
Abstract (Originalsprache)
There are several unsolved problems in federated learning, such as the security concerns and communication costs associated with it. Differential privacy (DP) offers effective privacy protection by introducing noise to parameters based on rigorous privacy definitions. However, excessive noise addition can potentially compromise the accuracy of the model. Another challenge in federated learning is the issue of high communication costs. Training large-scale federated models can be slow and expensive in terms of communication resources. To address this, various model pruning algorithms have been proposed. To address these challenges, this paper introduces a communication-efficient, privacy-preserving FL algorithm based on two-stage gradient pruning and differentiated differential privacy, named IsmDP-FL. The algorithm leverages a two-stage approach, incorporating gradient pruning and differentiated differential privacy. In the first stage, the trained model is subject to gradient pruning, followed by the addition of differential privacy to the important parameters selected after pruning. Non-important parameters are pruned by a certain ratio, and differentiated differential privacy is applied to the remaining parameters in each network layer. In the second stage, gradient pruning is performed during the upload to the server for aggregation, and the final result is returned to the client to complete the federated learning process. Extensive experiments demonstrate that the proposed method ensures a high communication efficiency, maintains the model privacy, and reduces the unnecessary use of the privacy budget.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computer Science and Engineering, Changchun University of Technology, Changchun 130012, China.
- AI Research Institute, Changchun University of Technology, Changchun 130012, China.
- School of Computer Science and Technology, Jilin University, Changchun 130012, China.
Förderung
- Science and Technology Research Planning Project of Jilin Provincial Department of Education in China
Zitieren
APA
Li, Y., Du, W., Han, L., Zhang, Z., & Liu, T. (2023). A Communication-Efficient, Privacy-Preserving Federated Learning Algorithm Based on Two-Stage Gradient Pruning and Differentiated Differential Privacy. Sensors, 23(23), 9305. https://doi.org/10.3390/s23239305
Vancouver
Li Y, Du W, Han L, Zhang Z, Liu T. A Communication-Efficient, Privacy-Preserving Federated Learning Algorithm Based on Two-Stage Gradient Pruning and Differentiated Differential Privacy. Sensors. 2023;23(23):9305. doi:10.3390/s23239305
BibTeX
@article{Li2023,
author = {Li, Yong and Du, Wei and Han, Liquan and Zhang, Zhenjian and Liu, Tongtong},
title = {A Communication-Efficient, Privacy-Preserving Federated Learning Algorithm Based on Two-Stage Gradient Pruning and Differentiated Differential Privacy},
journal = {Sensors},
year = {2023},
volume = {23},
number = {23},
pages = {9305},
doi = {10.3390/s23239305}
}
EUVIMED