EUVIMEDEUVIMED entdecken
Crossref

Rethinking LLM Deployment for Intent-Based Serving

Dimitrios Liakopoulos, Prasoon Sinha, Tianrui Hu, Myungjin Lee, Neeraja J. Yadwadkar

ACM SIGOPS Operating Systems Review · 13.07.2026 · 60 (1) · 50-59

DOI: 10.1145/3830422.3830428 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM SIGOPS Operating Systems Review, online erschienen am 13.07.2026. Verfasst von Dimitrios Liakopoulos und 4 weiteren Autor:innen. Die Arbeit nennt 48 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) expose a large deployment configuration space spanning parallelism and compression techniques, with each configuration introducing different tradeoffs in latency, memory consumption, cost, and output quality. Existing systems either rely on expensive profiling across deployment configurations or inefficiently utilize fragmented GPU resources in multi-tenant clusters. We present MaverIQ, an intent-based LLM inference serving system that automatically maps user intents to deployment configurations while minimizing operational cost for the provider. To reduce profiling overheads, MaverIQ introduces lightweight LLM fingerprints and analytical models that extrapolate latency and memory footprint from only a few observations. To efficiently utilize fragmented GPU resources, MaverIQ leverages our observation that, unlike training, unevenly distributing LLM layers across GPUs has little impact on inference latency. Our evaluation shows that MaverIQ reduces profiling cost by 7-15× compared to state-of-the-art baselines and reduces operational cost by 3.8-8.3× across diverse LLMs, traces, and loads while effectively meeting user intents. Our code is available at https://github.com/UT-SysML/MaverIQ.

Zitieren

APA

Liakopoulos, D., Sinha, P., Hu, T., Lee, M., & Yadwadkar, N. J. (2026). Rethinking LLM Deployment for Intent-Based Serving. ACM SIGOPS Operating Systems Review, 60(1), 50-59. https://doi.org/10.1145/3830422.3830428

Vancouver

Liakopoulos D, Sinha P, Hu T, Lee M, Yadwadkar NJ. Rethinking LLM Deployment for Intent-Based Serving. ACM SIGOPS Operating Systems Review. 2026;60(1):50-59. doi:10.1145/3830422.3830428

BibTeX
@article{Liakopoulos2026,
  author = {Liakopoulos, Dimitrios and Sinha, Prasoon and Hu, Tianrui and Lee, Myungjin and Yadwadkar, Neeraja J.},
  title = {Rethinking LLM Deployment for Intent-Based Serving},
  journal = {ACM SIGOPS Operating Systems Review},
  year = {2026},
  volume = {60},
  number = {1},
  pages = {50--59},
  doi = {10.1145/3830422.3830428}
}
Kostenlos starten →