Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks
IEEE Transactions on Wireless Communications · 2025-01 · 24 (1) · 643-658
DOI: 10.1109/twc.2024.3497923 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Wireless Communications, 2025-01. Verfasst von Xinyuan Zhang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 86-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: State Scholarship Fund of China Scholarship Council, National Research Foundation, Singapore, and the Infocomm Media Development Authority under its Future Communications Research and Development Program, Defence Science Organization (DSO) National Laboratories under the Artificial Intelligence (AI) Singapore Program.
Abstract (Originalsprache)
Generative Artificial Intelligenge (GAI) is revolutionizing the world with its unprecedented content creation ability. Large Language Model (LLM) is one of its most embraced branches. However, due to LLM’s substantial size and resource-intensive nature, it is cloud-hosted, raising concerns about privacy, usage limitations, and latency. In this paper, we propose to utilize ubiquitous distributed wireless edge computing resources for real-time LLM inference. Specifically, we introduce a novel LLM edge inference framework, incorporating batching and model quantization to ensure high throughput inference on resource-limited edge devices. Then, based on the architecture of transformer decoder-based LLMs, we formulate an edge inference optimization problem which is NP-hard, considering batch scheduling and joint allocation of communication and computation resources. The solution is the optimal throughput under edge resource constraints and heterogeneous user requirements on latency and accuracy. To solve this NP-hard problem, we develop an OT-GAH (Optimal Tree-search with Generalized Assignment Heuristics) algorithm with reasonable complexity and $\frac {1}{2}$ -approximation ratio. We first design the OT algorithm with online tree-pruning for single-edge-node multi-user case, which navigates the inference request selection within the tree structure to miximize throughput. We then consider the multi-edge-node case and propose the GAH algorithm, which recrusively invokes the OT in each node’s inference scheduling iteration. Simulation results demonstrate the superiority of OT-GAH batching over other benchmarks, revealing an over 45% time complexity reduction compared to brute-force searching.
Förderung
- State Scholarship Fund of China Scholarship Council
- National Research Foundation, Singapore, and the Infocomm Media Development Authority under its Future Communications Research and Development Program
- Defence Science Organization (DSO) National Laboratories under the Artificial Intelligence (AI) Singapore Program
- Singapore Ministry of Education (MOE) Tier 1
- Nanyang Technological University (NTU) Centre for Computational Technologies in Finance
- Ministry of Education, Singapore, under its Academic Research Fund Tier 2
- Singapore University of Technology and Design (SUTD) Kickstarter Initiative
Zitieren
APA
Zhang, X., Nie, J., Huang, Y., Xie, G., Xiong, Z., Liu, J., Niyato, D., & Shen, X. (2025). Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks. IEEE Transactions on Wireless Communications, 24(1), 643-658. https://doi.org/10.1109/twc.2024.3497923
Vancouver
Zhang X, Nie J, Huang Y, Xie G, Xiong Z, Liu J, et al. Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks. IEEE Transactions on Wireless Communications. 2025;24(1):643-658. doi:10.1109/twc.2024.3497923
BibTeX
@article{Zhang2025,
author = {Zhang, Xinyuan and Nie, Jiangtian and Huang, Yudong and Xie, Gaochang and Xiong, Zehui and Liu, Jiang and Niyato, Dusit and Shen, Xuemin},
title = {Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks},
journal = {IEEE Transactions on Wireless Communications},
year = {2025},
volume = {24},
number = {1},
pages = {643--658},
doi = {10.1109/twc.2024.3497923}
}
EUVIMED