EUVIMEDEUVIMED entdecken
Crossref

HST-MoE: Hierarchical Soft-Training for Multiexpert Federated Learning

Yongbo Yu, Zirui Xu, Fuxun Yu, Chenchen Liu, Xiang Chen

IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · 2026-11 · 45 (11) · 6028-6041

DOI: 10.1109/tcad.2026.3674445 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2026-11. Verfasst von Yongbo Yu und 4 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Federated learning (FL) with resource-constrained devices faces significant challenges from “stragglers,” computationally weak devices that delay synchronous parameter aggregation. This problem is particularly severe in mixture-of-experts (MoEs) architectures, where multiexpert computation imposes heavy burdens on edge devices. While removing stragglers accelerates training, it degrades model performance as these devices often hold critical information from unique local datasets. To resolve this dilemma, we propose HST-MoE, a hierarchical soft-training framework for multiexpert FL. Our approach introduces a two-level optimization: at the expert level, it performs dynamic rotation, training only subsets of experts per round; at the neuron level, it employs soft-training, updating fractions of parameters within active experts. This hierarchical design ensures all model components are trained effectively over time while maintaining balanced updates and stable convergence. Extensive experiments demonstrate that HST-MoE achieves up to $2.5\times $ training acceleration and improves convergence accuracy by up to 4.64% across various collaborative settings with stragglers, under both IID and nonidentically distributed (Non-IID) data distributions.

Zitieren

APA

Yu, Y., Xu, Z., Yu, F., Liu, C., & Chen, X. (2026). HST-MoE: Hierarchical Soft-Training for Multiexpert Federated Learning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 45(11), 6028-6041. https://doi.org/10.1109/tcad.2026.3674445

Vancouver

Yu Y, Xu Z, Yu F, Liu C, Chen X. HST-MoE: Hierarchical Soft-Training for Multiexpert Federated Learning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 2026;45(11):6028-6041. doi:10.1109/tcad.2026.3674445

BibTeX
@article{Yu2026,
  author = {Yu, Yongbo and Xu, Zirui and Yu, Fuxun and Liu, Chenchen and Chen, Xiang},
  title = {HST-MoE: Hierarchical Soft-Training for Multiexpert Federated Learning},
  journal = {IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems},
  year = {2026},
  volume = {45},
  number = {11},
  pages = {6028--6041},
  doi = {10.1109/tcad.2026.3674445}
}
Kostenlos starten →