WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching
ACM SIGMETRICS Performance Evaluation Review · 20.07.2026 · 54 (1) · 28-30
DOI: 10.1145/3820171.3806912 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM SIGMETRICS Performance Evaluation Review, online erschienen am 27.07.2026. Verfasst von Xiangchen Li und 9 weiteren Autor:innen. Die Arbeit nennt 9 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, Virginia Tech College of Engineering.
Abstract (Originalsprache)
LLM inference is increasingly initiated at edge devices but still executed in centralized GPU clusters. Distributed speculative edge serving offloads drafting to devices, yet existing systems scale poorly under heterogeneous, latency-sensitive workloads. We identify two bottlenecks: Wasted Drafting Time (WDT), where devices over-draft tokens later rejected by the verifier, and Verification Interference , where long verifications delay short requests within a micro-batch. We present WISP , a lossless distributed speculative serving system that combines dynamic drafting with SLO-aware verification batching. WISP uses a lightweight rejection predictor, a batch-latency estimator, and an interference-aware scheduler. Across simulation and prototype experiments, WISP improves system capacity by up to 2.1× over centralized serving and 4.1× over SLED , and improves system goodput by up to 1.94× and 3.7×, while reducing token-speed SLO violations under bursty multi-tenant workloads.
Förderung
- National Science Foundation
- Virginia Tech College of Engineering
Zitieren
APA
Li, X., Fan, J., Wang, Q., Spatharakis, D., Ghafouri, S., Vandierendonck, H., John, D., Ji, B., Butt, A. R., & Nikolopoulos, D. (2026). WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching. ACM SIGMETRICS Performance Evaluation Review, 54(1), 28-30. https://doi.org/10.1145/3820171.3806912
Vancouver
Li X, Fan J, Wang Q, Spatharakis D, Ghafouri S, Vandierendonck H, et al. WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching. ACM SIGMETRICS Performance Evaluation Review. 2026;54(1):28-30. doi:10.1145/3820171.3806912
BibTeX
@article{Li2026,
author = {Li, Xiangchen and Fan, Jiakun and Wang, Qingyuan and Spatharakis, Dimitrios and Ghafouri, Saeid and Vandierendonck, Hans and John, Deepu and Ji, Bo and Butt, Ali R. and Nikolopoulos, Dimitrios},
title = {WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching},
journal = {ACM SIGMETRICS Performance Evaluation Review},
year = {2026},
volume = {54},
number = {1},
pages = {28--30},
doi = {10.1145/3820171.3806912}
}
EUVIMED