EUVIMEDEUVIMED entdecken
Crossref

Comparing Load Balancing Algorithms in AI Inference Services

Yibo Li

Journal of Big Data and Computing · 2026-09 · 4 (3) · 44-57

DOI: 10.62517/jbdc.202601307 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Big Data and Computing, 2026-09. Verfasst von Yibo Li. Die Arbeit nennt 19 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The study focuses on load balancing in AI inference systems. The rapid growth of Artificial Intelligence (AI), in large language models (LLMs), has significantly impacted information systems. Classical algorithms (Round Robin, Least Outstanding Requests (LOR), and Power of Two Choices (P2C)) and AI-specific algorithms (Token-Aware and Prefix-Cache routing) are compared to determine their performance by applying a simulation-based approach, SimPy, under 25%, 60%, and 90% load conditions using response time, tail latency, throughput, and server utilization. Under low load conditions, all algorithms displayed similar results, while under moderate and high load conditions, AI-specific algorithms established remarkable results than classical algorithms.

Zitieren

APA

Li, Y. (2026). Comparing Load Balancing Algorithms in AI Inference Services. Journal of Big Data and Computing, 4(3), 44-57. https://doi.org/10.62517/jbdc.202601307

Vancouver

Li Y. Comparing Load Balancing Algorithms in AI Inference Services. Journal of Big Data and Computing. 2026;4(3):44-57. doi:10.62517/jbdc.202601307

BibTeX
@article{Li2026,
  author = {Li, Yibo},
  title = {Comparing Load Balancing Algorithms in AI Inference Services},
  journal = {Journal of Big Data and Computing},
  year = {2026},
  volume = {4},
  number = {3},
  pages = {44--57},
  doi = {10.62517/jbdc.202601307}
}
Kostenlos starten →