Learning When to Reason: Gating LLM Inference for Cost-Efficient Serverless Function Scheduling at Scale
Academic Journal of Applied Sciences · 09.06.2026 · 2 (1) · 39-45
DOI: 10.54097/gwmv0761 ↗Kurz gefasst
Fachartikel in der Zeitschrift Academic Journal of Applied Sciences, online erschienen am 09.06.2026. Verfasst von Megan Sullivan und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Serverless computing platforms face a fundamental tension when integrating large language model based scheduling agents: invoking full chain-of-thought reasoning for every function placement decision is computationally prohibitive, yet relying exclusively on lightweight heuristics cannot capture complex workload semantics. This paper presents GateLLM, a learned gating framework that selectively routes incoming serverless scheduling requests to either a lightweight fast-path handler or a deliberative large language model reasoning pipeline based on a compact feature vector extracted from request metadata. The gating classifier is a multilayer perceptron trained through a combination of offline oracle labeling and online reinforcement feedback derived from observed scheduling outcomes, enabling continuous adaptation to workload distribution shifts. Evaluated on a large-scale production function trace, GateLLM reduces total large language model inference cost by 61.3% relative to a full-reasoning baseline while incurring only a 2.1% degradation in average job completion time and a 0.25 percentage point increase in service level objective violation rate. Analysis demonstrates that over 68% of real-world scheduling events are structurally simple and resolvable without large language model involvement, establishing inference gating as an essential primitive for operationally viable intelligent schedulers in production cloud environments.
Zitieren
APA
Sullivan, M., He, B., & Evans, P. (2026). Learning When to Reason: Gating LLM Inference for Cost-Efficient Serverless Function Scheduling at Scale. Academic Journal of Applied Sciences, 2(1), 39-45. https://doi.org/10.54097/gwmv0761
Vancouver
Sullivan M, He B, Evans P. Learning When to Reason: Gating LLM Inference for Cost-Efficient Serverless Function Scheduling at Scale. Academic Journal of Applied Sciences. 2026;2(1):39-45. doi:10.54097/gwmv0761
BibTeX
@article{Sullivan2026,
author = {Sullivan, Megan and He, Boyang and Evans, Patrick},
title = {Learning When to Reason: Gating LLM Inference for Cost-Efficient Serverless Function Scheduling at Scale},
journal = {Academic Journal of Applied Sciences},
year = {2026},
volume = {2},
number = {1},
pages = {39--45},
doi = {10.54097/gwmv0761}
}
EUVIMED