A Provably Cost-Efficient Approach to Deploying MoE Inference Models at the Network Edge
IEEE Transactions on Services Computing · 2026-07 · 19 (4) · 2646-2659
DOI: 10.1109/tsc.2026.3707550 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Services Computing, 2026-07. Verfasst von Chao Wang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities.
Abstract (Originalsprache)
Foundation model inference is rapidly turning into a pay-per-request utility, yet today’s infrastructure still deploys experts as dedicated, always-on services. In Mixture-of-Experts models, sparse activation makes per-request execution paths highly dynamic, which amplifies provisioning uncertainty while creating opportunities for Inner-Service Resource Sharing (ISRS) across requests. This paper studies cost-efficient provisioning of MoE inference as a service over geo-distributed edge infrastructure, where a provider must satisfy the Service Level Agreement (SLA) constraints while minimizing the joint cost of compute, VRAM, and bandwidth. We formalize the Mixture-of-Experts Deployment (MoED) problem by explicitly modeling ISRS under dynamic routing and multicast communication. To solve MoED, we design an Expert Cost Loss (ECL) metric and a Sparsity-Aware Augmented Graph (SAG) that together guide ISRS-aware expert placement and bandwidth reservation, and we propose the ECL-based Expert Placement in SAG (ECL-EPS) algorithm. We prove that ECL-EPS achieves a logarithmic approximation ratio under practical assumptions. Extensive simulations show that ECL-EPS consistently reduces deployment cost and improves resource-sharing efficiency compared with the benchmarks.
Förderung
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities
Zitieren
APA
Wang, C., Zheng, D., Xing, H., Yang, C., Cao, X., Xu, J., & Teng, F. (2026). A Provably Cost-Efficient Approach to Deploying MoE Inference Models at the Network Edge. IEEE Transactions on Services Computing, 19(4), 2646-2659. https://doi.org/10.1109/tsc.2026.3707550
Vancouver
Wang C, Zheng D, Xing H, Yang C, Cao X, Xu J, et al. A Provably Cost-Efficient Approach to Deploying MoE Inference Models at the Network Edge. IEEE Transactions on Services Computing. 2026;19(4):2646-2659. doi:10.1109/tsc.2026.3707550
BibTeX
@article{Wang2026,
author = {Wang, Chao and Zheng, Danyang and Xing, Huanlai and Yang, Chen and Cao, Xiaojun and Xu, Jie and Teng, Fei},
title = {A Provably Cost-Efficient Approach to Deploying MoE Inference Models at the Network Edge},
journal = {IEEE Transactions on Services Computing},
year = {2026},
volume = {19},
number = {4},
pages = {2646--2659},
doi = {10.1109/tsc.2026.3707550}
}
EUVIMED