EUVIMEDEUVIMED entdecken
Crossref

Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems

Ruihao Li, Bagus Hanindhito, Sanjana Yadav, Qinzhe Wu, Krishna Kavi, Gayatri Mehta, Neeraja J. Yadwadkar, Lizy K. John

ACM Transactions on Architecture and Code Optimization · 30.09.2025 · 22 (3) · 1-26

DOI: 10.1145/3746231 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Architecture and Code Optimization, online erschienen am 19.09.2025. Verfasst von Ruihao Li und 7 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: NSF.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Driven by the increasing demands of machine learning, heterogeneous systems combining CPUs and GPUs have emerged as the dominant architecture for parallel computing in recent years. To optimize memory management and data transfer between CPUs and GPUs, Nvidia GPUs have introduced unified virtual memory ( UVM ) and pinned memory ( PM ) over the last decade. UVM can avoid explicit memory copies and potentially overlap GPU kernel computations with CPU-GPU data transfer. PM ensures that data with high locality remains in the main memory, preventing it from being paged out. In addition to these two techniques, asynchronous memory copy ( Async Memcpy ) was introduced recently in Nvidia GPUs to improve the CPU-GPU pipeline further. By utilizing Async Memcpy , the data transfer from GPU global memory to shared memory can be overlapped with GPU computations, adding an additional stage to the CPU-GPU data transfer pipeline. A thorough performance analysis of how Async Memcpy affects the current UVM and PM CPU-GPU data transfer scheme is desired. In this article, we provide performance implications of the combined effect of UVM , PM , and Async Memcpy , exploring which applications benefit from which combination of these features. We implement all these features on a suite of 25 workloads, including microbenchmarks and realworld applications. We observe an average performance gain of 24% when utilizing UVM and a 34% gain when employing PM on realworld applications, compared to not applying any data transfer optimization techniques. The performance benefits of Async Memcpy vary across different workloads. For workloads featuring extensive shared memory usage and high compute density (e.g., kmeans and lud ), Async Memcpy delivers around a 20% performance improvement over using UVM or PM alone. In other workloads like knn , we note a 20% performance degradation when using Async Memcpy . Furthermore, we conduct an in-depth investigation of the GPU kernel using performance counters to uncover the root causes of performance differences among various data transfer models. We also perform sensitivity analyses to examine how the number of blocks and threads, as well as the L1-cache/shared memory partitioning, impact performance. We explore future research directions aimed at enhancing the data transfer pipeline by overlapping memory allocation with data transfer and computation across GPU kernels.

Förderung

  • NSF

Zitieren

APA

Li, R., Hanindhito, B., Yadav, S., Wu, Q., Kavi, K., Mehta, G., Yadwadkar, N. J., & John, L. K. (2025). Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems. ACM Transactions on Architecture and Code Optimization, 22(3), 1-26. https://doi.org/10.1145/3746231

Vancouver

Li R, Hanindhito B, Yadav S, Wu Q, Kavi K, Mehta G, et al. Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems. ACM Transactions on Architecture and Code Optimization. 2025;22(3):1-26. doi:10.1145/3746231

BibTeX
@article{Li2025,
  author = {Li, Ruihao and Hanindhito, Bagus and Yadav, Sanjana and Wu, Qinzhe and Kavi, Krishna and Mehta, Gayatri and Yadwadkar, Neeraja J. and John, Lizy K.},
  title = {Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems},
  journal = {ACM Transactions on Architecture and Code Optimization},
  year = {2025},
  volume = {22},
  number = {3},
  pages = {1--26},
  doi = {10.1145/3746231}
}
Kostenlos starten →