EUVIMEDEUVIMED entdecken
Crossref

A Provably Cost-Efficient Approach to Deploying MoE Inference Models at the Network Edge

Chao Wang, Danyang Zheng, Huanlai Xing, Chen Yang, Xiaojun Cao, Jie Xu, Fei Teng

IEEE Transactions on Services Computing · 2026-07 · 19 (4) · 2646-2659

DOI: 10.1109/tsc.2026.3707550 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Services Computing, 2026-07. Verfasst von Chao Wang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Foundation model inference is rapidly turning into a pay-per-request utility, yet today’s infrastructure still deploys experts as dedicated, always-on services. In Mixture-of-Experts models, sparse activation makes per-request execution paths highly dynamic, which amplifies provisioning uncertainty while creating opportunities for Inner-Service Resource Sharing (ISRS) across requests. This paper studies cost-efficient provisioning of MoE inference as a service over geo-distributed edge infrastructure, where a provider must satisfy the Service Level Agreement (SLA) constraints while minimizing the joint cost of compute, VRAM, and bandwidth. We formalize the Mixture-of-Experts Deployment (MoED) problem by explicitly modeling ISRS under dynamic routing and multicast communication. To solve MoED, we design an Expert Cost Loss (ECL) metric and a Sparsity-Aware Augmented Graph (SAG) that together guide ISRS-aware expert placement and bandwidth reservation, and we propose the ECL-based Expert Placement in SAG (ECL-EPS) algorithm. We prove that ECL-EPS achieves a logarithmic approximation ratio under practical assumptions. Extensive simulations show that ECL-EPS consistently reduces deployment cost and improves resource-sharing efficiency compared with the benchmarks.

Förderung

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities

Zitieren

APA

Wang, C., Zheng, D., Xing, H., Yang, C., Cao, X., Xu, J., & Teng, F. (2026). A Provably Cost-Efficient Approach to Deploying MoE Inference Models at the Network Edge. IEEE Transactions on Services Computing, 19(4), 2646-2659. https://doi.org/10.1109/tsc.2026.3707550

Vancouver

Wang C, Zheng D, Xing H, Yang C, Cao X, Xu J, et al. A Provably Cost-Efficient Approach to Deploying MoE Inference Models at the Network Edge. IEEE Transactions on Services Computing. 2026;19(4):2646-2659. doi:10.1109/tsc.2026.3707550

BibTeX
@article{Wang2026,
  author = {Wang, Chao and Zheng, Danyang and Xing, Huanlai and Yang, Chen and Cao, Xiaojun and Xu, Jie and Teng, Fei},
  title = {A Provably Cost-Efficient Approach to Deploying MoE Inference Models at the Network Edge},
  journal = {IEEE Transactions on Services Computing},
  year = {2026},
  volume = {19},
  number = {4},
  pages = {2646--2659},
  doi = {10.1109/tsc.2026.3707550}
}
Kostenlos starten →