HST-MoE: Hierarchical Soft-Training for Multiexpert Federated Learning
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · 2026-11 · 45 (11) · 6028-6041
DOI: 10.1109/tcad.2026.3674445 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2026-11. Verfasst von Yongbo Yu und 4 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben.
Abstract (Originalsprache)
Federated learning (FL) with resource-constrained devices faces significant challenges from “stragglers,” computationally weak devices that delay synchronous parameter aggregation. This problem is particularly severe in mixture-of-experts (MoEs) architectures, where multiexpert computation imposes heavy burdens on edge devices. While removing stragglers accelerates training, it degrades model performance as these devices often hold critical information from unique local datasets. To resolve this dilemma, we propose HST-MoE, a hierarchical soft-training framework for multiexpert FL. Our approach introduces a two-level optimization: at the expert level, it performs dynamic rotation, training only subsets of experts per round; at the neuron level, it employs soft-training, updating fractions of parameters within active experts. This hierarchical design ensures all model components are trained effectively over time while maintaining balanced updates and stable convergence. Extensive experiments demonstrate that HST-MoE achieves up to $2.5\times $ training acceleration and improves convergence accuracy by up to 4.64% across various collaborative settings with stragglers, under both IID and nonidentically distributed (Non-IID) data distributions.
Zitieren
APA
Yu, Y., Xu, Z., Yu, F., Liu, C., & Chen, X. (2026). HST-MoE: Hierarchical Soft-Training for Multiexpert Federated Learning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 45(11), 6028-6041. https://doi.org/10.1109/tcad.2026.3674445
Vancouver
Yu Y, Xu Z, Yu F, Liu C, Chen X. HST-MoE: Hierarchical Soft-Training for Multiexpert Federated Learning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 2026;45(11):6028-6041. doi:10.1109/tcad.2026.3674445
BibTeX
@article{Yu2026,
author = {Yu, Yongbo and Xu, Zirui and Yu, Fuxun and Liu, Chenchen and Chen, Xiang},
title = {HST-MoE: Hierarchical Soft-Training for Multiexpert Federated Learning},
journal = {IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems},
year = {2026},
volume = {45},
number = {11},
pages = {6028--6041},
doi = {10.1109/tcad.2026.3674445}
}
EUVIMED