EUVIMEDEUVIMED entdecken
Crossref

Heterogeneous Federated Learning Driven by Multi-Knowledge Distillation

Bin Xu, Longgang Cheng, Qing Wen, Zhensheng Zou, Xiaoxuan Hu, Zhenjiang Dong, Jin Qi

IEEE Transactions on Mobile Computing · 2025-12 · 24 (12) · 13048-13061

DOI: 10.1109/tmc.2025.3586921 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2025-12. Verfasst von Bin Xu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 38 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In a fully heterogeneous federated learning environment, the client has significant differences in model structure and local data distribution (Non-IID), and the joint learning of the client model is blocked due to the limited communication content available for interaction in a fully heterogeneous scenario. In this context, the global knowledge constructed by the server through the simple aggregation of the client logits is essentially a fuzzy representation containing a lot of noise and information loss, which is difficult to effectively guide the client model update. To solve these problems, this paper proposes a heterogeneous federated learning framework (FedMkd) based on multi-knowledge distillation fusion to cope with multiple challenges in heterogeneous environments. The FedMkd framework uses a class-grained logits interaction architecture (CLIA) and introduces an efficient knowledge sharing mechanism. It innovatively integrates two knowledge distillation methods: 1) Temperature-Adaptive Knowledge Distillation (TAKD), which provides differentiated temperatures for teacher and student models by adaptively adjusting the distillation temperature, maximizing knowledge transfer between them; 2) Class-related Knowledge Distillation (CRKD), which introduces batch-level sample correlation loss to reduce over-reliance on specific samples or classes and improve the model’s understanding of overall data features. We conducted a large number of experiments on four public datasets. The results show that in a variety of data and model heterogeneous scenarios, FedMkd still performs better than the comparison method when the communication overhead is reduced by more than one order of magnitude.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Xu, B., Cheng, L., Wen, Q., Zou, Z., Hu, X., Dong, Z., & Qi, J. (2025). Heterogeneous Federated Learning Driven by Multi-Knowledge Distillation. IEEE Transactions on Mobile Computing, 24(12), 13048-13061. https://doi.org/10.1109/tmc.2025.3586921

Vancouver

Xu B, Cheng L, Wen Q, Zou Z, Hu X, Dong Z, et al. Heterogeneous Federated Learning Driven by Multi-Knowledge Distillation. IEEE Transactions on Mobile Computing. 2025;24(12):13048-13061. doi:10.1109/tmc.2025.3586921

BibTeX
@article{Xu2025,
  author = {Xu, Bin and Cheng, Longgang and Wen, Qing and Zou, Zhensheng and Hu, Xiaoxuan and Dong, Zhenjiang and Qi, Jin},
  title = {Heterogeneous Federated Learning Driven by Multi-Knowledge Distillation},
  journal = {IEEE Transactions on Mobile Computing},
  year = {2025},
  volume = {24},
  number = {12},
  pages = {13048--13061},
  doi = {10.1109/tmc.2025.3586921}
}
Kostenlos starten →