EUVIMEDEUVIMED entdecken
Crossref

FedCD: Towards Consolidated Distillation for Heterogeneous Federated Learning

Yichen Li, Hang Su, Huifa Li, Haolin Yang, Xinlin Zhuang, Haochen Xue, Haozhao Wang, Imran Razzak

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (28) · 23256-23264

DOI: 10.1609/aaai.v40i28.39494 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yichen Li und 7 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Knowledge Distillation (KD) serves as an effective approach to addressing heterogeneity issues in Federated Learning (FL), leveraging additional datasets to align local and global models better. There are two primary distillation paradigms: feature-based distillation, which utilizes intermediate-layer features of the network, and logit-based distillation, which employs the final layer's logit outputs. However, existing studies often select distillation methods based on intuitive and empirical evidence when facing different heterogeneous settings, neglecting the intrinsic relationship between distillation paradigms and heterogeneity. This oversight may result in suboptimal federated knowledge distillation performance under heterogeneous conditions. In this paper, we propose the Consolidated Distillation for Heterogeneous Federated Learning - FedCD that balances knowledge representations from both feature-based and logit-based distillation to enhance performance. Specifically, to address the misalignment between knowledge conveyed by features and logits, we aggregate features from different layers via cross-layer attention to preserve semantic knowledge, followed by distribution modeling using Gaussian Mixture Models. This process strengthens knowledge distillation by constraining the transformation of different network layers' features under a consolidated distribution, thereby mitigating impacts from both data and model heterogeneity. Extensive experiments demonstrate that FedCD outperforms state-of-the-art methods by over 10.72% and validate the effectiveness of our approach.

Zitieren

APA

Li, Y., Su, H., Li, H., Yang, H., Zhuang, X., Xue, H., Wang, H., & Razzak, I. (2026). FedCD: Towards Consolidated Distillation for Heterogeneous Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(28), 23256-23264. https://doi.org/10.1609/aaai.v40i28.39494

Vancouver

Li Y, Su H, Li H, Yang H, Zhuang X, Xue H, et al. FedCD: Towards Consolidated Distillation for Heterogeneous Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(28):23256-23264. doi:10.1609/aaai.v40i28.39494

BibTeX
@article{Li2026,
  author = {Li, Yichen and Su, Hang and Li, Huifa and Yang, Haolin and Zhuang, Xinlin and Xue, Haochen and Wang, Haozhao and Razzak, Imran},
  title = {FedCD: Towards Consolidated Distillation for Heterogeneous Federated Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {28},
  pages = {23256--23264},
  doi = {10.1609/aaai.v40i28.39494}
}
Kostenlos starten →