pFedKA: Personalized Federated Learning via Knowledge Distillation with Dual Attention Mechanism
Computers · 21.11.2025 · 14 (12) · 504
DOI: 10.3390/computers14120504 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computers, online erschienen am 21.11.2025. Verfasst von Yuanhao Jin und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Key Research and Development Program of Heilongjiang.
Abstract (Originalsprache)
Federated learning in heterogeneous data scenarios faces two key challenges. First, the conflict between global models and local personalization complicates knowledge transfer and leads to feature misalignment, hindering effective personalization for clients. Second, the lack of dynamic adaptation in standard federated learning makes it difficult to handle highly heterogeneous and changing client data, reducing the global model’s generalization ability. To address these issues, this paper proposes pFedKA, a personalized federated learning framework integrating knowledge distillation and a dual-attention mechanism. On the client-side, a cross-attention module dynamically aligns global and local feature spaces using adaptive temperature coefficients to mitigate feature misalignment. On the server-side, a Gated Recurrent Unit-based attention network adaptively adjusts aggregation weights using cross-round historical states, providing more robust aggregation than static averaging in heterogeneous settings. Experimental results on CIFAR-10, CIFAR-100, and Shakespeare datasets demonstrate that pFedKA converges faster and with greater stability in heterogeneous scenarios. Furthermore, it significantly improves personalization accuracy compared to state-of-the-art personalized federated learning methods. Additionally, we demonstrate privacy guarantees by integrating pFedKA with DP-SGD, showing comparable privacy protection to FedAvg while maintaining high personalization accuracy.
Förderung
- National Natural Science Foundation of China
- Key Research and Development Program of Heilongjiang
Zitieren
APA
Jin, Y., Zhang, K., Ma, C., Cheng, X., Zhang, L., & Zhang, H. (2025). pFedKA: Personalized Federated Learning via Knowledge Distillation with Dual Attention Mechanism. Computers, 14(12), 504. https://doi.org/10.3390/computers14120504
Vancouver
Jin Y, Zhang K, Ma C, Cheng X, Zhang L, Zhang H. pFedKA: Personalized Federated Learning via Knowledge Distillation with Dual Attention Mechanism. Computers. 2025;14(12):504. doi:10.3390/computers14120504
BibTeX
@article{Jin2025,
author = {Jin, Yuanhao and Zhang, Kaiqi and Ma, Chao and Cheng, Xinxin and Zhang, Luogang and Zhang, Hongguo},
title = {pFedKA: Personalized Federated Learning via Knowledge Distillation with Dual Attention Mechanism},
journal = {Computers},
year = {2025},
volume = {14},
number = {12},
pages = {504},
doi = {10.3390/computers14120504}
}
EUVIMED