Comparing Load Balancing Algorithms in AI Inference Services
Journal of Big Data and Computing · 2026-09 · 4 (3) · 44-57
DOI: 10.62517/jbdc.202601307 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Big Data and Computing, 2026-09. Verfasst von Yibo Li. Die Arbeit nennt 19 Literaturangaben.
Abstract (Originalsprache)
The study focuses on load balancing in AI inference systems. The rapid growth of Artificial Intelligence (AI), in large language models (LLMs), has significantly impacted information systems. Classical algorithms (Round Robin, Least Outstanding Requests (LOR), and Power of Two Choices (P2C)) and AI-specific algorithms (Token-Aware and Prefix-Cache routing) are compared to determine their performance by applying a simulation-based approach, SimPy, under 25%, 60%, and 90% load conditions using response time, tail latency, throughput, and server utilization. Under low load conditions, all algorithms displayed similar results, while under moderate and high load conditions, AI-specific algorithms established remarkable results than classical algorithms.
Zitieren
APA
Li, Y. (2026). Comparing Load Balancing Algorithms in AI Inference Services. Journal of Big Data and Computing, 4(3), 44-57. https://doi.org/10.62517/jbdc.202601307
Vancouver
Li Y. Comparing Load Balancing Algorithms in AI Inference Services. Journal of Big Data and Computing. 2026;4(3):44-57. doi:10.62517/jbdc.202601307
BibTeX
@article{Li2026,
author = {Li, Yibo},
title = {Comparing Load Balancing Algorithms in AI Inference Services},
journal = {Journal of Big Data and Computing},
year = {2026},
volume = {4},
number = {3},
pages = {44--57},
doi = {10.62517/jbdc.202601307}
}
EUVIMED