EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

XSLA: AN SLA-AWARE HETEROGENEOUS XPU SCHEDULING FRAMEWORK FOR EDGE AI INFERENCE

TingJie Chen, WenBin Shang, ZiXuan Guo

World Journal of Information Technology · 01.01.2024 · 2 (2) · 48-59

DOI: 10.61784/wjit4125 ↗

Kurz gefasst

Fachartikel in der Zeitschrift World Journal of Information Technology, online erschienen am 01.01.2024. Verfasst von TingJie Chen und 2 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Edge AI deployments increasingly aggregate several classes of accelerator — embedded GPUs, neural processing units, FPGAs, dataflow ASICs and ARM CPUs — into a single "XPU" pool, while the requests they serve arrive with heterogeneous service-level agreements (SLAs). Using published measurements of 15,625 architectures on six real accelerator platforms, we show that one fixed model varies in latency by up to 51.9x across XPU classes, that 30.1% of architectures cannot execute on the Edge TPU at all, and that no accelerator is simultaneously best for latency, energy and accuracy. Which accelerator to use and which model variant to run are therefore one decision, not two. We present XSLA, which co-selects variant and accelerator per request. XSLA (i) shrinks the candidate set with a per-accelerator Pareto test we prove is lossless, (ii) chooses among deadline-feasible candidates by a drift-plus-penalty rule whose quality weight adapts to measured backlog, and (iii) estimates backlog per SLA class so latency-critical traffic is never queued behind best-effort work. In a measurement-driven simulator with 6 XPU classes, 3 tasks and 24 real variants, XSLA holds SLA violation at 0.0% up to 9,000 req/s, where an accuracy-greedy model-less policy misses 47.0% of deadlines and fixed-model EDF misses 99.0%. It raises deadline-constrained accuracy yield by 14.4–81.4% over the strongest baseline while using 37.4–58.4% less energy per request, lifting pool utilisation from 0.26 to 0.99.

Zitieren

APA

Chen, T., Shang, W., & Guo, Z. (2024). XSLA: AN SLA-AWARE HETEROGENEOUS XPU SCHEDULING FRAMEWORK FOR EDGE AI INFERENCE. World Journal of Information Technology, 2(2), 48-59. https://doi.org/10.61784/wjit4125

Vancouver

Chen T, Shang W, Guo Z. XSLA: AN SLA-AWARE HETEROGENEOUS XPU SCHEDULING FRAMEWORK FOR EDGE AI INFERENCE. World Journal of Information Technology. 2024;2(2):48-59. doi:10.61784/wjit4125

BibTeX
@article{Chen2024,
  author = {Chen, TingJie and Shang, WenBin and Guo, ZiXuan},
  title = {XSLA: AN SLA-AWARE HETEROGENEOUS XPU SCHEDULING FRAMEWORK FOR EDGE AI INFERENCE},
  journal = {World Journal of Information Technology},
  year = {2024},
  volume = {2},
  number = {2},
  pages = {48--59},
  doi = {10.61784/wjit4125}
}
Kostenlos starten →