EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Efficient multi-tenant LoRA serving via SGMV-specific operator autotuning

Chao Zhang, Gang Zhou, Teng Fu, Jicang Lu

Journal of King Saud University Computer and Information Sciences · 2026-08 · 38 (6)

DOI: 10.1007/s44443-026-00937-7 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of King Saud University Computer and Information Sciences, online erschienen am 15.06.2026. Verfasst von Chao Zhang und 3 weiteren Autor:innen. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Low-Rank Adaptation (LoRA) has become a mainstream technique for fine-tuning large language models (LLMs) on downstream tasks due to its memory efficiency. However, traditional inference architectures suffer from computational redundancy and low GPU utilization when serving multi-tenant LoRA models. To address these inefficiencies, this paper presents GD2O, an SGMV-specific operator autotuning framework for multi-tenant LoRA serving. GD2O uses a graph representation to organize the construction space of the non-standard Segmented Gather Matrix-Vector Multiplication (SGMV) operator and applies a hardware-aware Markov-style stochastic search to select tiling, caching, and virtual-thread scheduling configurations. The search is used as a practical heuristic for efficient exploration, not as a formal guarantee of global optimality. The text generation experiments on Llama-2 and two GPU architectures (NVIDIA A100 and RTX 4090) demonstrate that GD2O achieves a throughput improvement of 1.29–1.52×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times $$\end{document} in multi-tenant LoRA serving.

Zitieren

APA

Zhang, C., Zhou, G., Fu, T., & Lu, J. (2026). Efficient multi-tenant LoRA serving via SGMV-specific operator autotuning. Journal of King Saud University Computer and Information Sciences, 38(6). https://doi.org/10.1007/s44443-026-00937-7

Vancouver

Zhang C, Zhou G, Fu T, Lu J. Efficient multi-tenant LoRA serving via SGMV-specific operator autotuning. Journal of King Saud University Computer and Information Sciences. 2026;38(6). doi:10.1007/s44443-026-00937-7

BibTeX
@article{Zhang2026,
  author = {Zhang, Chao and Zhou, Gang and Fu, Teng and Lu, Jicang},
  title = {Efficient multi-tenant LoRA serving via SGMV-specific operator autotuning},
  journal = {Journal of King Saud University Computer and Information Sciences},
  year = {2026},
  volume = {38},
  number = {6},
  doi = {10.1007/s44443-026-00937-7}
}
Kostenlos starten →