EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training With Non-IID Private Data

Sohei Itahara, Takayuki Nishio, Yusuke Koda, Masahiro Morikura, Koji Yamamoto

IEEE Transactions on Mobile Computing · 01.01.2023 · 22 (1) · 191-205

DOI: 10.1109/tmc.2021.3070013 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, erschienen am 01.01.2023. Verfasst von Sohei Itahara und 4 weiteren Autor:innen. Die Arbeit wurde bisher 310-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: JSPS KAKENHI, JST PRESTO, KDDI Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study develops a federated learning (FL) framework overcoming largely incremental communication costs due to model sizes in typical frameworks without compromising model performance. To this end, based on the idea of leveraging an unlabeled open dataset, we propose a distillation-based semi-supervised FL (DS-FL) algorithm that exchanges the outputs of local models among mobile devices, instead of model parameter exchange employed by the typical frameworks. In DS-FL, the communication cost depends only on the output dimensions of the models and does not scale up according to the model size. The exchanged model outputs are used to label each sample of the open dataset, which creates an additionally labeled dataset. Based on the new dataset, local models are further trained, and model performance is enhanced owing to the data augmentation effect. We further highlight that in DS-FL, the heterogeneity of the devices’ dataset leads to ambiguous of each data sample and lowing of the training convergence. To prevent this, we propose entropy reduction averaging, where the aggregated model outputs are intentionally sharpened. Moreover, extensive experiments show that DS-FL reduces communication costs up to 99 percent relative to those of the FL benchmark while achieving similar or higher classification accuracy.

Förderung

  • JSPS KAKENHI
  • JST PRESTO
  • KDDI Foundation

Zitieren

APA

Itahara, S., Nishio, T., Koda, Y., Morikura, M., & Yamamoto, K. (2023). Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training With Non-IID Private Data. IEEE Transactions on Mobile Computing, 22(1), 191-205. https://doi.org/10.1109/tmc.2021.3070013

Vancouver

Itahara S, Nishio T, Koda Y, Morikura M, Yamamoto K. Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training With Non-IID Private Data. IEEE Transactions on Mobile Computing. 2023;22(1):191-205. doi:10.1109/tmc.2021.3070013

BibTeX
@article{Itahara2023,
  author = {Itahara, Sohei and Nishio, Takayuki and Koda, Yusuke and Morikura, Masahiro and Yamamoto, Koji},
  title = {Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training With Non-IID Private Data},
  journal = {IEEE Transactions on Mobile Computing},
  year = {2023},
  volume = {22},
  number = {1},
  pages = {191--205},
  doi = {10.1109/tmc.2021.3070013}
}
Kostenlos starten →