EUVIMEDEUVIMED entdecken
Crossref

Learning Critically: Selective Self-Distillation in Federated Learning on Non-IID Data

Yuting He, Yiqiang Chen, XiaoDong Yang, Hanchao Yu, Yi-Hua Huang, Yang Gu

IEEE Transactions on Big Data · 2024-12 · 10 (6) · 789-800

DOI: 10.1109/tbdata.2022.3189703 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Big Data, 2024-12. Verfasst von Yuting He und 5 weiteren Autor:innen. Die Arbeit wurde bisher 33-mal zitiert (Crossref) und nennt 51 Literaturangaben. Gefördert von: National Key Research and Development Plan of China, National Natural Science Foundation of China, Beijing Municipal Science and Technology Commission.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Federated learning (FL) enables multiple clients to collaboratively train a global model while keeping local data decentralized. Data heterogeneity (non-IID) across clients has imposed significant challenges to FL, which makes local models re-optimize towards their own local optima and forget the global knowledge, resulting in performance degradation and convergence slowdown. Many existing works have attempted to address the non-IID issue by adding an extra global-model-based regularizing item to the local training but without an adaption scheme, which is not efficient enough to achieve high performance with deep learning models. In this paper, we propose a Selective Self-Distillation method for Federated learning (FedSSD), which imposes adaptive constraints on the local updates by self-distilling the global model's knowledge and selectively weighting it by evaluating the credibility at both the class and sample level. The convergence guarantee of FedSSD is theoretically analyzed and extensive experiments are conducted on three public benchmark datasets, which demonstrates that FedSSD achieves better generalization and robustness in fewer communication rounds, compared with other state-of-the-art FL methods.

Förderung

  • National Key Research and Development Plan of China
  • National Natural Science Foundation of China
  • Beijing Municipal Science and Technology Commission
  • Jinan S&T Bureau
  • Youth Innovation Promotion Association of the Chinese Academy of Sciences
  • Science, and Technology Service Network Initiative
  • Chinese Academy of Sciences

Zitieren

APA

He, Y., Chen, Y., Yang, X., Yu, H., Huang, Y., & Gu, Y. (2024). Learning Critically: Selective Self-Distillation in Federated Learning on Non-IID Data. IEEE Transactions on Big Data, 10(6), 789-800. https://doi.org/10.1109/tbdata.2022.3189703

Vancouver

He Y, Chen Y, Yang X, Yu H, Huang Y, Gu Y. Learning Critically: Selective Self-Distillation in Federated Learning on Non-IID Data. IEEE Transactions on Big Data. 2024;10(6):789-800. doi:10.1109/tbdata.2022.3189703

BibTeX
@article{He2024,
  author = {He, Yuting and Chen, Yiqiang and Yang, XiaoDong and Yu, Hanchao and Huang, Yi-Hua and Gu, Yang},
  title = {Learning Critically: Selective Self-Distillation in Federated Learning on Non-IID Data},
  journal = {IEEE Transactions on Big Data},
  year = {2024},
  volume = {10},
  number = {6},
  pages = {789--800},
  doi = {10.1109/tbdata.2022.3189703}
}
Kostenlos starten →