EUVIMEDEUVIMED entdecken
Crossref

A Survey on Inference Optimization Techniques for Mixture of Experts Models

Jiacheng Liu, Peng Tang, Wenfeng Wang, Yuhang Ren, Xiaofeng Hou, Pheng Ann Heng, Minyi Guo, Chao Li

ACM Computing Surveys · 31.07.2026 · 58 (10) · 1-37

DOI: 10.1145/3794845 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 09.03.2026. Verfasst von Jiacheng Liu und 7 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 242 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Research Grants Council of the Hong Kong Special Administrative Region, China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The emergence of large-scale Mixture of Experts (MoE) models represents a significant advancement in artificial intelligence, offering larger model capacity and computational efficiency through conditional computation. However, deploying and running inference on these models presents significant challenges in computational resources, latency, and energy efficiency. This comprehensive survey analyzes optimization techniques for MoE models across the entire system stack. We first establish a taxonomical framework that categorizes optimization approaches into model-level, system-level, and hardware-level optimizations. At the model level, we examine architectural innovations including efficient expert design, attention mechanisms, various compression techniques such as pruning, quantization, and knowledge distillation, as well as algorithm improvement including dynamic routing strategies and expert merging methods. At the system level, we investigate distributed computing approaches, load balancing mechanisms, and efficient scheduling algorithms that enable scalable deployment. Furthermore, we delve into hardware-specific optimizations and co-design strategies that maximize throughput and energy efficiency. This survey provides both a structured overview of existing solutions and identifies key challenges and promising research directions in MoE inference optimization. To facilitate ongoing updates and the sharing of cutting-edge advances in MoE inference optimization research, we have established a repository accessible at https://github.com/MoE-Inf/awesome-moe-inference/ .

Förderung

  • National Natural Science Foundation of China
  • Research Grants Council of the Hong Kong Special Administrative Region, China

Zitieren

APA

Liu, J., Tang, P., Wang, W., Ren, Y., Hou, X., Heng, P. A., Guo, M., & Li, C. (2026). A Survey on Inference Optimization Techniques for Mixture of Experts Models. ACM Computing Surveys, 58(10), 1-37. https://doi.org/10.1145/3794845

Vancouver

Liu J, Tang P, Wang W, Ren Y, Hou X, Heng PA, et al. A Survey on Inference Optimization Techniques for Mixture of Experts Models. ACM Computing Surveys. 2026;58(10):1-37. doi:10.1145/3794845

BibTeX
@article{Liu2026,
  author = {Liu, Jiacheng and Tang, Peng and Wang, Wenfeng and Ren, Yuhang and Hou, Xiaofeng and Heng, Pheng Ann and Guo, Minyi and Li, Chao},
  title = {A Survey on Inference Optimization Techniques for Mixture of Experts Models},
  journal = {ACM Computing Surveys},
  year = {2026},
  volume = {58},
  number = {10},
  pages = {1--37},
  doi = {10.1145/3794845}
}
Kostenlos starten →