Efficient LLM Inference Over Heterogeneous Edge Networks With Speculative Decoding
IEEE Transactions on Communications · 2026 · 74 · 11189-11204
DOI: 10.1109/tcomm.2026.3712530 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Communications, 2026. Verfasst von Bingjie Zhu und 4 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Gefördert von: Engineering and Physical Sciences Research Council.
Abstract (Originalsprache)
Existing edge-based LLM inference systems typically adopt autoregressive decoding (AD), which generates only one token per forward pass. This iterative process, compounded by the limited computational resources of edge nodes, results in high serving latency and constrains the system’s ability to support multiple users under growing demands. Speculative decoding (SD) has emerged as a promising approach for accelerating inference by enabling multi-token generation per forward pass. However, existing SD-based edge inference systems primarily lack an integrated framework that jointly considers SD, batching, pipeline execution, and wireless communication, which may hinder the effectiveness of SD in providing low-latency LLM services in edge networks. To address this challenge, we develop a unified analytical framework for SD-based LLM inference systems over heterogeneous edge networks, where the draft and target models are deployed across different edge nodes and a pipeline-parallel serving mechanism is employed to overlap draft speculation and target verification across inference tasks. Building on this framework, we derive a comprehensive serving latency model that captures wireless communication latency, fronthaul transmission latency, and inference latency. Then, we formulate a joint optimization problem for speculation length, task batching, and wireless communication resource allocation to minimize total serving latency. To address this problem, we derive the closed-form solutions for wireless communication resource allocation, and develop a dynamic programming algorithm for joint batching and speculation control strategies. Experimental results demonstrate that the proposed framework achieves lower serving latency compared to AD-based serving systems. In addition, the proposed joint optimization method delivers up to 44.1% latency reduction compared to benchmark schemes.
Förderung
- Engineering and Physical Sciences Research Council
Zitieren
APA
Zhu, B., Chen, Z., Zhao, L., Shin, H., & Nallanathan, A. (2026). Efficient LLM Inference Over Heterogeneous Edge Networks With Speculative Decoding. IEEE Transactions on Communications, 74, 11189-11204. https://doi.org/10.1109/tcomm.2026.3712530
Vancouver
Zhu B, Chen Z, Zhao L, Shin H, Nallanathan A. Efficient LLM Inference Over Heterogeneous Edge Networks With Speculative Decoding. IEEE Transactions on Communications. 2026;74:11189-11204. doi:10.1109/tcomm.2026.3712530
BibTeX
@article{Zhu2026,
author = {Zhu, Bingjie and Chen, Zhixiong and Zhao, Liqiang and Shin, Hyundong and Nallanathan, Arumugam},
title = {Efficient LLM Inference Over Heterogeneous Edge Networks With Speculative Decoding},
journal = {IEEE Transactions on Communications},
year = {2026},
volume = {74},
pages = {11189--11204},
doi = {10.1109/tcomm.2026.3712530}
}
EUVIMED