EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

pFedKA: Personalized Federated Learning via Knowledge Distillation with Dual Attention Mechanism

Yuanhao Jin, Kaiqi Zhang, Chao Ma, Xinxin Cheng, Luogang Zhang, Hongguo Zhang

Computers · 21.11.2025 · 14 (12) · 504

DOI: 10.3390/computers14120504 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computers, online erschienen am 21.11.2025. Verfasst von Yuanhao Jin und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Key Research and Development Program of Heilongjiang.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Federated learning in heterogeneous data scenarios faces two key challenges. First, the conflict between global models and local personalization complicates knowledge transfer and leads to feature misalignment, hindering effective personalization for clients. Second, the lack of dynamic adaptation in standard federated learning makes it difficult to handle highly heterogeneous and changing client data, reducing the global model’s generalization ability. To address these issues, this paper proposes pFedKA, a personalized federated learning framework integrating knowledge distillation and a dual-attention mechanism. On the client-side, a cross-attention module dynamically aligns global and local feature spaces using adaptive temperature coefficients to mitigate feature misalignment. On the server-side, a Gated Recurrent Unit-based attention network adaptively adjusts aggregation weights using cross-round historical states, providing more robust aggregation than static averaging in heterogeneous settings. Experimental results on CIFAR-10, CIFAR-100, and Shakespeare datasets demonstrate that pFedKA converges faster and with greater stability in heterogeneous scenarios. Furthermore, it significantly improves personalization accuracy compared to state-of-the-art personalized federated learning methods. Additionally, we demonstrate privacy guarantees by integrating pFedKA with DP-SGD, showing comparable privacy protection to FedAvg while maintaining high personalization accuracy.

Förderung

  • National Natural Science Foundation of China
  • Key Research and Development Program of Heilongjiang

Zitieren

APA

Jin, Y., Zhang, K., Ma, C., Cheng, X., Zhang, L., & Zhang, H. (2025). pFedKA: Personalized Federated Learning via Knowledge Distillation with Dual Attention Mechanism. Computers, 14(12), 504. https://doi.org/10.3390/computers14120504

Vancouver

Jin Y, Zhang K, Ma C, Cheng X, Zhang L, Zhang H. pFedKA: Personalized Federated Learning via Knowledge Distillation with Dual Attention Mechanism. Computers. 2025;14(12):504. doi:10.3390/computers14120504

BibTeX
@article{Jin2025,
  author = {Jin, Yuanhao and Zhang, Kaiqi and Ma, Chao and Cheng, Xinxin and Zhang, Luogang and Zhang, Hongguo},
  title = {pFedKA: Personalized Federated Learning via Knowledge Distillation with Dual Attention Mechanism},
  journal = {Computers},
  year = {2025},
  volume = {14},
  number = {12},
  pages = {504},
  doi = {10.3390/computers14120504}
}
Kostenlos starten →