EUVIMEDEUVIMED entdecken
Crossref

EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices

Rongjie Yi, Liwei Guo, Shiyun Wei, Ao Zhou, Shangguang Wang, Mengwei Xu

IEEE Transactions on Mobile Computing · 2025-08 · 24 (8) · 7059-7073

DOI: 10.1109/tmc.2025.3546466 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2025-08. Verfasst von Rongjie Yi und 5 weiteren Autor:innen. Die Arbeit wurde bisher 53-mal zitiert (Crossref) und nennt 95 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, Sichuan Science and Technology Plan.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) such as GPTs and Mixtral-8x7B have revolutionized machine intelligence due to their exceptional abilities in generic ML tasks. Transiting LLMs from datacenters to edge devices brings benefits like better privacy and availability, but is challenged by their massive parameter size and thus unbearable runtime costs. To this end, we present EdgeMoE, an on-device inference engine for mixture-of-expert (MoE) LLMs – a popular form of sparse LLM that scales its parameter size with almost constant computing complexity. EdgeMoE achieves both memory- and compute-efficiency by partitioning the model into the storage hierarchy: non-expert weights are held in device memory; while expert weights are held on external storage and fetched to memory only when activated. This design is motivated by a key observation that expert weights are bulky but infrequently used due to sparse activation. To further reduce the expert I/O swapping overhead, EdgeMoE incorporates two novel techniques: (1) expert-wise bitwidth adaptation that reduces the expert sizes with tolerable accuracy loss; (2) expert preloading that predicts the activated experts ahead of time and preloads it with the compute-I/O pipeline. On popular MoE LLMs and edge devices, EdgeMoE showcase significant memory savings and speedup over competitive baselines.

Förderung

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities
  • Sichuan Science and Technology Plan
  • Unveiling and Leading Project

Zitieren

APA

Yi, R., Guo, L., Wei, S., Zhou, A., Wang, S., & Xu, M. (2025). EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices. IEEE Transactions on Mobile Computing, 24(8), 7059-7073. https://doi.org/10.1109/tmc.2025.3546466

Vancouver

Yi R, Guo L, Wei S, Zhou A, Wang S, Xu M. EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices. IEEE Transactions on Mobile Computing. 2025;24(8):7059-7073. doi:10.1109/tmc.2025.3546466

BibTeX
@article{Yi2025,
  author = {Yi, Rongjie and Guo, Liwei and Wei, Shiyun and Zhou, Ao and Wang, Shangguang and Xu, Mengwei},
  title = {EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices},
  journal = {IEEE Transactions on Mobile Computing},
  year = {2025},
  volume = {24},
  number = {8},
  pages = {7059--7073},
  doi = {10.1109/tmc.2025.3546466}
}
Kostenlos starten →