Efficient Mixture-of-Experts Model Inference at the Edge via Adaptive Expert Merging
IEEE Transactions on Networking · 2026 · 34 · 5942-5953
DOI: 10.1109/ton.2026.3704584 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Networking, 2026. Verfasst von Ruirui Zhang und 7 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China.
Abstract (Originalsprache)
This paper studies the Mixture-of-Experts (MoE) model inference problem at the edge via the adaptive expert merging technique. To fill the gap between the large size of MoE models and the limited hardware resources at the edge, existing works primarily focus on static merging policies and overlook the dynamic and heterogeneous nature of edge environments. This paper addresses these gaps by proposing a flexible framework for adaptive expert merging and online inference task offloading on edge servers. Our contributions include 1) a novel approach allowing each edge server to adaptively merge experts based on its resources and task pReferences, 2) an online inference task offloading algorithm with a bounded competitive ratio for pre-determined merging policies, and 3) an online algorithm for joint optimization of task offloading and expert merging, which ensures timely model updates and assignment of tasks to specialized servers. Experimental results on five datasets demonstrate that our approach reduces the overall weighted cost by at least 16% compared to baseline methods.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
Zitieren
APA
Zhang, R., Zou, Y., Li, P., Chen, F., Li, Y., Cheng, X., Dressler, F., & Yu, D. (2026). Efficient Mixture-of-Experts Model Inference at the Edge via Adaptive Expert Merging. IEEE Transactions on Networking, 34, 5942-5953. https://doi.org/10.1109/ton.2026.3704584
Vancouver
Zhang R, Zou Y, Li P, Chen F, Li Y, Cheng X, et al. Efficient Mixture-of-Experts Model Inference at the Edge via Adaptive Expert Merging. IEEE Transactions on Networking. 2026;34:5942-5953. doi:10.1109/ton.2026.3704584
BibTeX
@article{Zhang2026,
author = {Zhang, Ruirui and Zou, Yifei and Li, Peng and Chen, Fahao and Li, Yupeng and Cheng, Xiuzhen and Dressler, Falko and Yu, Dongxiao},
title = {Efficient Mixture-of-Experts Model Inference at the Edge via Adaptive Expert Merging},
journal = {IEEE Transactions on Networking},
year = {2026},
volume = {34},
pages = {5942--5953},
doi = {10.1109/ton.2026.3704584}
}
EUVIMED