EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

SAI: Virtualizing Shared Memory of GPU for AI workload acceleration

Hanqing Li, Tiejun Li, Sheng Ma, Jianming Zhang, Yuhan Tang, Siqing Fu, Jixuan Tang, Lizhou Wu

ACM Transactions on Design Automation of Electronic Systems · 31.01.2027 · 32 (1) · 1-23

DOI: 10.1145/3819826 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Design Automation of Electronic Systems, online erschienen am 03.11.2026. Verfasst von Hanqing Li und 7 weiteren Autor:innen. Die Arbeit nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

AI workloads increasingly demand high parallelism and efficient on-chip memory utilization on modern GPUs. However, the limited capacity of shared memory often constrains thread-level parallelism, while portions of the L2 cache remain underutilized. In this work, we propose SAI , a mechanism that virtualizes shared memory into the L2 cache to improve GPU performance for AI applications. SAI dynamically activates virtualization based on runtime resource usage, enabling additional CTAs with minimal architectural modifications. We further introduce an L2 cache management strategy that integrates associativity-based virtual page allocation and a replacement information table, reducing page-swapping overhead while preserving L2 cache performance. Experimental results demonstrate that SAI achieves an 18.6% performance improvement over the baseline design and outperforms SMILE-opt, our reproduced state-of-the-art baseline, by 8.8%. Moreover, SAI captures 60.2% of the performance gain delivered by the idealized Double SMEM design while reducing energy consumption to 87.7% of the baseline. These results highlight the effectiveness of SAI in enhancing thread-level parallelism and optimizing on-chip memory for AI workloads.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Li, H., Li, T., Ma, S., Zhang, J., Tang, Y., Fu, S., Tang, J., & Wu, L. (2027). SAI: Virtualizing Shared Memory of GPU for AI workload acceleration. ACM Transactions on Design Automation of Electronic Systems, 32(1), 1-23. https://doi.org/10.1145/3819826

Vancouver

Li H, Li T, Ma S, Zhang J, Tang Y, Fu S, et al. SAI: Virtualizing Shared Memory of GPU for AI workload acceleration. ACM Transactions on Design Automation of Electronic Systems. 2027;32(1):1-23. doi:10.1145/3819826

BibTeX
@article{Li2027,
  author = {Li, Hanqing and Li, Tiejun and Ma, Sheng and Zhang, Jianming and Tang, Yuhan and Fu, Siqing and Tang, Jixuan and Wu, Lizhou},
  title = {SAI: Virtualizing Shared Memory of GPU for AI workload acceleration},
  journal = {ACM Transactions on Design Automation of Electronic Systems},
  year = {2027},
  volume = {32},
  number = {1},
  pages = {1--23},
  doi = {10.1145/3819826}
}
Kostenlos starten →