EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Personalized Data-Free Knowledge Distillation for Federated Learning under Heterogeneous Models and Data

Jingke Tu, Lei Yang, Chao Ma, Weigang Wu

ACM Transactions on Knowledge Discovery from Data · 31.08.2026 · 20 (7) · 1-27

DOI: 10.1145/3831243 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Knowledge Discovery from Data, online erschienen am 20.08.2026. Verfasst von Jingke Tu und 3 weiteren Autor:innen. Die Arbeit nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Guangdong Basic and Applied Basic Research Foundation, Hong Kong RGC Theme-based Research Scheme, NSFC and Hong Kong RGC Collaborative Research Scheme.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Knowledge Distillation (KD) is considered as an efficient way to replace the parameter averaging in federated learning, aiming to handle the clients with heterogeneous model architectures. Relying on the prepared distillation datasets across clients and the server, KD may encounter impractical difficulties in real-world implementations. Existing works explore the data-free KD in federated learning, which generates the distillation datasets on-site. However, the distillation datasets with global data distribution generated by these state-of-the-art schemes cannot be adapted to local non-IID data. In this article, we propose a new Personalized Data-Free Knowledge Distillation, namely PDKD, for federated learning under heterogeneous models and data. PDKD solves the problem of model drift caused by the inconsistent distribution of distillation datasets and the local data by generating personalized distillation datasets for each client while protecting client data privacy. In addition, we design a distillation dataset update scheme that maximizes the difference between teacher and client outputs on distillation datasets to accomplish deeper knowledge transfer. Furthermore, in order to accomplish the co-evolution of the teacher model and the clients’ model, PDKD incorporates a mutual distillation scheme. Numerous experiments show that PDKD significantly outperforms several state-of-the-art algorithms, with an 18% improvement in prediction accuracy and has a much lower communication cost than the compared algorithms.

Förderung

  • Guangdong Basic and Applied Basic Research Foundation
  • Hong Kong RGC Theme-based Research Scheme
  • NSFC and Hong Kong RGC Collaborative Research Scheme

Zitieren

APA

Tu, J., Yang, L., Ma, C., & Wu, W. (2026). Personalized Data-Free Knowledge Distillation for Federated Learning under Heterogeneous Models and Data. ACM Transactions on Knowledge Discovery from Data, 20(7), 1-27. https://doi.org/10.1145/3831243

Vancouver

Tu J, Yang L, Ma C, Wu W. Personalized Data-Free Knowledge Distillation for Federated Learning under Heterogeneous Models and Data. ACM Transactions on Knowledge Discovery from Data. 2026;20(7):1-27. doi:10.1145/3831243

BibTeX
@article{Tu2026,
  author = {Tu, Jingke and Yang, Lei and Ma, Chao and Wu, Weigang},
  title = {Personalized Data-Free Knowledge Distillation for Federated Learning under Heterogeneous Models and Data},
  journal = {ACM Transactions on Knowledge Discovery from Data},
  year = {2026},
  volume = {20},
  number = {7},
  pages = {1--27},
  doi = {10.1145/3831243}
}
Kostenlos starten →