EUVIMEDEUVIMED entdecken
Crossref

Towards Data-Independent Knowledge Transfer in Model-Heterogeneous Federated Learning

Jie Zhang, Song Guo, Jingcai Guo, Deze Zeng, Jingren Zhou, Albert Y. Zomaya

IEEE Transactions on Computers · 2023-10 · 72 (10) · 2888-2901

DOI: 10.1109/tc.2023.3272801 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Computers, 2023-10. Verfasst von Jie Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 66-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: Key-Area Research and Development Program of Guangdong Province, Hong Kong RGC Research Impact Fund, General Research Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Federated Distillation (FD) extends classic Federated Learning (FL) to a more general training framework that enables model-heterogeneous collaborative learning by Knowledge Distillation (KD) across multiple clients and the server. However, existing KD-based algorithms usually require a set of shared input samples for each client to produce soft-prediction for distillation. Worse still, such a manual selection is accompanied by careful deliberations or prior information on clients’ private data distribution, which is not in line with the privacy-preserving characteristic of classic FL. In this paper, we propose a novel training framework to achieve data-independent knowledge transfer by properly designing a distributed generative adversarial network (GAN) between the server and clients that can synthesize shared feature representations to facilitate the FD training. Specifically, we deploy a generator on the server and reuse each local model as a federated discriminator to form a lightweight efficient distributed GAN that can automatically synthesize simulated global feature representations for distillation. Moreover, since the synthesized feature representations are usually more faithful and homologous with global data distribution, faster and better training convergence can be obtained. Extensive experiments on different tasks and heterogeneous models demonstrate the effectiveness of the proposed framework on model accuracy and communication overhead.

Förderung

  • Key-Area Research and Development Program of Guangdong Province
  • Hong Kong RGC Research Impact Fund
  • General Research Fund
  • National Natural Science Foundation of China
  • PolyU Internal Fund
  • Shenzhen Science and Technology Innovation Commission
  • Areas of Excellence Scheme

Zitieren

APA

Zhang, J., Guo, S., Guo, J., Zeng, D., Zhou, J., & Zomaya, A. Y. (2023). Towards Data-Independent Knowledge Transfer in Model-Heterogeneous Federated Learning. IEEE Transactions on Computers, 72(10), 2888-2901. https://doi.org/10.1109/tc.2023.3272801

Vancouver

Zhang J, Guo S, Guo J, Zeng D, Zhou J, Zomaya AY. Towards Data-Independent Knowledge Transfer in Model-Heterogeneous Federated Learning. IEEE Transactions on Computers. 2023;72(10):2888-2901. doi:10.1109/tc.2023.3272801

BibTeX
@article{Zhang2023,
  author = {Zhang, Jie and Guo, Song and Guo, Jingcai and Zeng, Deze and Zhou, Jingren and Zomaya, Albert Y.},
  title = {Towards Data-Independent Knowledge Transfer in Model-Heterogeneous Federated Learning},
  journal = {IEEE Transactions on Computers},
  year = {2023},
  volume = {72},
  number = {10},
  pages = {2888--2901},
  doi = {10.1109/tc.2023.3272801}
}
Kostenlos starten →