EUVIMEDEUVIMED entdecken
Crossref

Straggler-Resilient Differentially Private Decentralized Learning

Yauhen Yakimenka, Chung-Wei Weng, Hsuan-Yin Lin, Eirik Rosnes, Jörg Kliewer

IEEE Journal on Selected Areas in Information Theory · 2024 · 5 · 407-423

DOI: 10.1109/jsait.2024.3400995 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Journal on Selected Areas in Information Theory, 2024. Verfasst von Yauhen Yakimenka und 4 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: Experimental Infrastructure for Exploration of Exascale Computing, Research Council of Norway, U.S. NSF.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We consider the straggler problem in decentralized learning over a logical ring while preserving user data privacy. Especially, we extend the recently proposed framework of differential privacy (DP) amplification by decentralization by Cyffers and Bellet to include overall training latency—comprising both computation and communication latency. Analytical results on both the convergence speed and the DP level are derived for both a skipping scheme (which ignores the stragglers after a timeout) and a baseline scheme that waits for each node to finish before the training continues. A trade-off between overall training latency, accuracy, and privacy, parameterized by the timeout of the skipping scheme, is identified and empirically validated for logistic regression on a real-world dataset and for image classification using the MNIST and CIFAR-10 datasets.

Förderung

  • Experimental Infrastructure for Exploration of Exascale Computing
  • Research Council of Norway
  • U.S. NSF

Zitieren

APA

Yakimenka, Y., Weng, C., Lin, H., Rosnes, E., & Kliewer, J. (2024). Straggler-Resilient Differentially Private Decentralized Learning. IEEE Journal on Selected Areas in Information Theory, 5, 407-423. https://doi.org/10.1109/jsait.2024.3400995

Vancouver

Yakimenka Y, Weng C, Lin H, Rosnes E, Kliewer J. Straggler-Resilient Differentially Private Decentralized Learning. IEEE Journal on Selected Areas in Information Theory. 2024;5:407-423. doi:10.1109/jsait.2024.3400995

BibTeX
@article{Yakimenka2024,
  author = {Yakimenka, Yauhen and Weng, Chung-Wei and Lin, Hsuan-Yin and Rosnes, Eirik and Kliewer, Jörg},
  title = {Straggler-Resilient Differentially Private Decentralized Learning},
  journal = {IEEE Journal on Selected Areas in Information Theory},
  year = {2024},
  volume = {5},
  pages = {407--423},
  doi = {10.1109/jsait.2024.3400995}
}
Kostenlos starten →