EUVIMEDEUVIMED entdecken
Crossref

Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks

Xinyuan Zhang, Jiangtian Nie, Yudong Huang, Gaochang Xie, Zehui Xiong, Jiang Liu, Dusit Niyato, Xuemin Shen

IEEE Transactions on Wireless Communications · 2025-01 · 24 (1) · 643-658

DOI: 10.1109/twc.2024.3497923 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Wireless Communications, 2025-01. Verfasst von Xinyuan Zhang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 86-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: State Scholarship Fund of China Scholarship Council, National Research Foundation, Singapore, and the Infocomm Media Development Authority under its Future Communications Research and Development Program, Defence Science Organization (DSO) National Laboratories under the Artificial Intelligence (AI) Singapore Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Generative Artificial Intelligenge (GAI) is revolutionizing the world with its unprecedented content creation ability. Large Language Model (LLM) is one of its most embraced branches. However, due to LLM’s substantial size and resource-intensive nature, it is cloud-hosted, raising concerns about privacy, usage limitations, and latency. In this paper, we propose to utilize ubiquitous distributed wireless edge computing resources for real-time LLM inference. Specifically, we introduce a novel LLM edge inference framework, incorporating batching and model quantization to ensure high throughput inference on resource-limited edge devices. Then, based on the architecture of transformer decoder-based LLMs, we formulate an edge inference optimization problem which is NP-hard, considering batch scheduling and joint allocation of communication and computation resources. The solution is the optimal throughput under edge resource constraints and heterogeneous user requirements on latency and accuracy. To solve this NP-hard problem, we develop an OT-GAH (Optimal Tree-search with Generalized Assignment Heuristics) algorithm with reasonable complexity and $\frac {1}{2}$ -approximation ratio. We first design the OT algorithm with online tree-pruning for single-edge-node multi-user case, which navigates the inference request selection within the tree structure to miximize throughput. We then consider the multi-edge-node case and propose the GAH algorithm, which recrusively invokes the OT in each node’s inference scheduling iteration. Simulation results demonstrate the superiority of OT-GAH batching over other benchmarks, revealing an over 45% time complexity reduction compared to brute-force searching.

Förderung

  • State Scholarship Fund of China Scholarship Council
  • National Research Foundation, Singapore, and the Infocomm Media Development Authority under its Future Communications Research and Development Program
  • Defence Science Organization (DSO) National Laboratories under the Artificial Intelligence (AI) Singapore Program
  • Singapore Ministry of Education (MOE) Tier 1
  • Nanyang Technological University (NTU) Centre for Computational Technologies in Finance
  • Ministry of Education, Singapore, under its Academic Research Fund Tier 2
  • Singapore University of Technology and Design (SUTD) Kickstarter Initiative

Zitieren

APA

Zhang, X., Nie, J., Huang, Y., Xie, G., Xiong, Z., Liu, J., Niyato, D., & Shen, X. (2025). Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks. IEEE Transactions on Wireless Communications, 24(1), 643-658. https://doi.org/10.1109/twc.2024.3497923

Vancouver

Zhang X, Nie J, Huang Y, Xie G, Xiong Z, Liu J, et al. Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks. IEEE Transactions on Wireless Communications. 2025;24(1):643-658. doi:10.1109/twc.2024.3497923

BibTeX
@article{Zhang2025,
  author = {Zhang, Xinyuan and Nie, Jiangtian and Huang, Yudong and Xie, Gaochang and Xiong, Zehui and Liu, Jiang and Niyato, Dusit and Shen, Xuemin},
  title = {Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks},
  journal = {IEEE Transactions on Wireless Communications},
  year = {2025},
  volume = {24},
  number = {1},
  pages = {643--658},
  doi = {10.1109/twc.2024.3497923}
}
Kostenlos starten →