pFedMoE: Data-Level Personalization With Mixture of Experts in Model-Heterogeneous Personalized Federated Learning
IEEE Transactions on Knowledge and Data Engineering · 2026-03 · 38 (3) · 1905-1918
DOI: 10.1109/tkde.2026.3656194 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Knowledge and Data Engineering, 2026-03. Verfasst von Liping Yi und 4 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 75 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Ministry of Education - Singapore, Academic Research Fund Tier 1.
Abstract (Originalsprache)
With growing client diversity, model-heterogeneous personalized federated learning (MHPFL) supports collaboration over structure-heterogeneous client models. However, existing MHPFL methods only achieve client-level personalization but ignore inherent discrepancies within each client’s different data samples, leading to limited model performance. To this end, we propose a novel model-heterogeneous p ersonalized Fed erated learning with M ixture o f E xperts ( pFedMoE ) to achieve a fine-grained data-level personalization. As the first work that incorporates MoE in MHPFL, it introduces three innovations: (1) Different clients hold heterogeneous local models, we add a small proxy global homogeneous feature extractor shared by clients for knowledge exchange. (2) To achieve a fine-grained data-level personalization, we construct a personalized local MoE for each client: a local expert (local heterogeneous client model’s feature extractor), a global expert (global proxy homogeneous feature extractor), and a local personalized gating network, which dynamically balances the generalization and personalization of the local model at the data sample level. (3) We customize a lightweight linear gating network to capture the generalized and personalized data characteristics of each local data sample. We theoretically prove its $\mathcal {O}(1/T)$ O ( 1 / T ) convergence rate. Experiments on 3 benchmark image datasets, 1 real-world image dataset and 1 real-world text dataset against 9 baselines demonstrate its state-of-the-art model accuracy with up to 2.79% accuracy improvement while saving up to 43.12% computational overheads and keeping satisfactory communication costs.
Förderung
- National Natural Science Foundation of China
- Ministry of Education - Singapore
- Academic Research Fund Tier 1
Zitieren
APA
Yi, L., Yu, H., Wang, G., Liu, X., & Hu, Q. (2026). pFedMoE: Data-Level Personalization With Mixture of Experts in Model-Heterogeneous Personalized Federated Learning. IEEE Transactions on Knowledge and Data Engineering, 38(3), 1905-1918. https://doi.org/10.1109/tkde.2026.3656194
Vancouver
Yi L, Yu H, Wang G, Liu X, Hu Q. pFedMoE: Data-Level Personalization With Mixture of Experts in Model-Heterogeneous Personalized Federated Learning. IEEE Transactions on Knowledge and Data Engineering. 2026;38(3):1905-1918. doi:10.1109/tkde.2026.3656194
BibTeX
@article{Yi2026,
author = {Yi, Liping and Yu, Han and Wang, Gang and Liu, Xiaoguang and Hu, Qinghua},
title = {pFedMoE: Data-Level Personalization With Mixture of Experts in Model-Heterogeneous Personalized Federated Learning},
journal = {IEEE Transactions on Knowledge and Data Engineering},
year = {2026},
volume = {38},
number = {3},
pages = {1905--1918},
doi = {10.1109/tkde.2026.3656194}
}
EUVIMED