Edge Inference for Large Language Models With Pipeline Parallelism and Batching
IEEE Transactions on Communications · 2026 · 74 · 8390-8406
DOI: 10.1109/tcomm.2026.3686718 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Communications, 2026. Verfasst von Jie Jiang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Gefördert von: Engineering and Physical Sciences Research Council (EPSRC), U.K.
Abstract (Originalsprache)
Edge computing enables distributed inference for computation-intensive applications. However, the autoregressive nature and large model size of large language models (LLMs) pose challenges for their deployment in wireless edge networks. Existing edge inference methods mainly assume an equivalent delay for each token generation step, which fails to capture the dynamic computational and memory overhead incurred during the decoding process. This paper proposes a latency-sensitive wireless edge inference framework for LLMs, where tasks are grouped into multiple batches and processed in parallel by partitioning the LLM into multiple pipeline stages across heterogeneous edge GPUs. An accurate latency model is established, where the latency of each token generation step increases during the autoregressive generation process. Based on this model, the end-to-end inference latency is minimized, which is formulated as a joint optimization problem of bandwidth allocation, model partitioning, and batch scheduling subject to heterogeneous GPU memory constraints. To solve this NP-hard problem with coupled variables, we develop a polynomial-time alternating optimization algorithm that iteratively optimizes model partitioning and batch scheduling via dynamic programming. The closed-form solutions of wireless bandwidth allocation are derived. Extensive simulations show that our approach reduces latency by up to 42.1% versus state-of-the-art baselines across diverse edge scenarios.
Förderung
- Engineering and Physical Sciences Research Council (EPSRC), U.K
Zitieren
APA
Jiang, J., Chen, Z., Shin, H., & Nallanathan, A. (2026). Edge Inference for Large Language Models With Pipeline Parallelism and Batching. IEEE Transactions on Communications, 74, 8390-8406. https://doi.org/10.1109/tcomm.2026.3686718
Vancouver
Jiang J, Chen Z, Shin H, Nallanathan A. Edge Inference for Large Language Models With Pipeline Parallelism and Batching. IEEE Transactions on Communications. 2026;74:8390-8406. doi:10.1109/tcomm.2026.3686718
BibTeX
@article{Jiang2026,
author = {Jiang, Jie and Chen, Zhixiong and Shin, Hyundong and Nallanathan, Arumugam},
title = {Edge Inference for Large Language Models With Pipeline Parallelism and Batching},
journal = {IEEE Transactions on Communications},
year = {2026},
volume = {74},
pages = {8390--8406},
doi = {10.1109/tcomm.2026.3686718}
}
EUVIMED