EUVIMEDEUVIMED entdecken
Crossref

Mil: Cost-Guided Minimum Makespan Scheduling for Applications of Multiple LLMs

Jingzhi Fang, Yanyan Shen, Yue Wang, Lei Chen

Proceedings of the VLDB Endowment · 01.05.2026 · 19 (9) · 1893-1906

DOI: 10.14778/3819518.3819522 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 19.08.2026. Verfasst von Jingzhi Fang und 3 weiteren Autor:innen. Die Arbeit nennt 58 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-LLM applications calling multiple LLMs per request are emerging. An important scenario is running these applications offline on a request set. This work aims to minimize the offline inference makespan of these applications to save time and cost. Specifically, we study minimum-makespan scheduling of multi-LLM applications (the MLAS problem), which requires GPU allocation, LLM parallelism selection, and LLM execution orchestration. MLAS is NP-hard, and it differs from existing multi-model frameworks and job scheduling problems due to LLMs' unique properties (e.g., high memory demand, complex inference behavior), the offline inference setting, and relaxed execution precedence constraints. There is no existing work on MLAS and simple rules cannot handle all the problem instances. We propose a framework, Mil, for MLAS with three major components: (1) processing functions estimating LLM processing rates by output length sampling, inference process simulation, and per-generation-iteration latency estimation; (2) a greedy method that finds a good schedule with a theoretical guarantee on a simplified problem instance; (3) a runtime adjustment mechanism reducing GPU idleness. Experiments on various applications (ensembling, routing, chain summary, mixed) show that Mil can achieve up to 3.4× end-to-end speedups over current practice.

Zitieren

APA

Fang, J., Shen, Y., Wang, Y., & Chen, L. (2026). Mil: Cost-Guided Minimum Makespan Scheduling for Applications of Multiple LLMs. Proceedings of the VLDB Endowment, 19(9), 1893-1906. https://doi.org/10.14778/3819518.3819522

Vancouver

Fang J, Shen Y, Wang Y, Chen L. Mil: Cost-Guided Minimum Makespan Scheduling for Applications of Multiple LLMs. Proceedings of the VLDB Endowment. 2026;19(9):1893-1906. doi:10.14778/3819518.3819522

BibTeX
@article{Fang2026,
  author = {Fang, Jingzhi and Shen, Yanyan and Wang, Yue and Chen, Lei},
  title = {Mil: Cost-Guided Minimum Makespan Scheduling for Applications of Multiple LLMs},
  journal = {Proceedings of the VLDB Endowment},
  year = {2026},
  volume = {19},
  number = {9},
  pages = {1893--1906},
  doi = {10.14778/3819518.3819522}
}
Kostenlos starten →