SAI: Virtualizing Shared Memory of GPU for AI workload acceleration
ACM Transactions on Design Automation of Electronic Systems · 31.01.2027 · 32 (1) · 1-23
DOI: 10.1145/3819826 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Design Automation of Electronic Systems, online erschienen am 03.11.2026. Verfasst von Hanqing Li und 7 weiteren Autor:innen. Die Arbeit nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
AI workloads increasingly demand high parallelism and efficient on-chip memory utilization on modern GPUs. However, the limited capacity of shared memory often constrains thread-level parallelism, while portions of the L2 cache remain underutilized. In this work, we propose SAI , a mechanism that virtualizes shared memory into the L2 cache to improve GPU performance for AI applications. SAI dynamically activates virtualization based on runtime resource usage, enabling additional CTAs with minimal architectural modifications. We further introduce an L2 cache management strategy that integrates associativity-based virtual page allocation and a replacement information table, reducing page-swapping overhead while preserving L2 cache performance. Experimental results demonstrate that SAI achieves an 18.6% performance improvement over the baseline design and outperforms SMILE-opt, our reproduced state-of-the-art baseline, by 8.8%. Moreover, SAI captures 60.2% of the performance gain delivered by the idealized Double SMEM design while reducing energy consumption to 87.7% of the baseline. These results highlight the effectiveness of SAI in enhancing thread-level parallelism and optimizing on-chip memory for AI workloads.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Li, H., Li, T., Ma, S., Zhang, J., Tang, Y., Fu, S., Tang, J., & Wu, L. (2027). SAI: Virtualizing Shared Memory of GPU for AI workload acceleration. ACM Transactions on Design Automation of Electronic Systems, 32(1), 1-23. https://doi.org/10.1145/3819826
Vancouver
Li H, Li T, Ma S, Zhang J, Tang Y, Fu S, et al. SAI: Virtualizing Shared Memory of GPU for AI workload acceleration. ACM Transactions on Design Automation of Electronic Systems. 2027;32(1):1-23. doi:10.1145/3819826
BibTeX
@article{Li2027,
author = {Li, Hanqing and Li, Tiejun and Ma, Sheng and Zhang, Jianming and Tang, Yuhan and Fu, Siqing and Tang, Jixuan and Wu, Lizhou},
title = {SAI: Virtualizing Shared Memory of GPU for AI workload acceleration},
journal = {ACM Transactions on Design Automation of Electronic Systems},
year = {2027},
volume = {32},
number = {1},
pages = {1--23},
doi = {10.1145/3819826}
}
EUVIMED