EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Benchmarking GPU Tensor Cores on General Matrix Multiplication Kernels through CUTLASS

Xuanteng Huang, Xianwei Zhang, Panfei Yang, Nong Xiao

Applied Sciences · 06.12.2023 · 13 (24) · 13022

DOI: 10.3390/app132413022 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 06.12.2023. Verfasst von Xuanteng Huang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Major Program of Guangdong Basic and Applied Research, Funding by Science and Technology Projects in Guangzhou.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

GPUs have been broadly used to accelerate big data analytics, scientific computing and machine intelligence. Particularly, matrix multiplication and convolution are two principal operations that use a large proportion of steps in modern data analysis and deep neural networks. These performance-critical operations are often offloaded to the GPU to obtain substantial improvements in end-to-end latency. In addition, multifarious workload characteristics and complicated processing phases in big data demand a customizable yet performant operator library. To this end, GPU vendors, including NVIDIA and AMD, have proposed template and composable GPU operator libraries to conduct specific computations on certain types of low-precision data elements. We formalize a set of benchmarks via CUTLASS, NVIDIA’s templated library that provides high-performance and hierarchically designed kernels. The benchmarking results show that, with the necessary fine tuning, hardware-level ASICs like tensor cores could dramatically boost performance in specific operations like GEMM offloading to modern GPUs.

Förderung

  • National Natural Science Foundation of China
  • Major Program of Guangdong Basic and Applied Research
  • Funding by Science and Technology Projects in Guangzhou
  • Open Project of China Electronic Product Reliability and Environmental Testing Research Institute (CEPREI)

Zitieren

APA

Huang, X., Zhang, X., Yang, P., & Xiao, N. (2023). Benchmarking GPU Tensor Cores on General Matrix Multiplication Kernels through CUTLASS. Applied Sciences, 13(24), 13022. https://doi.org/10.3390/app132413022

Vancouver

Huang X, Zhang X, Yang P, Xiao N. Benchmarking GPU Tensor Cores on General Matrix Multiplication Kernels through CUTLASS. Applied Sciences. 2023;13(24):13022. doi:10.3390/app132413022

BibTeX
@article{Huang2023,
  author = {Huang, Xuanteng and Zhang, Xianwei and Yang, Panfei and Xiao, Nong},
  title = {Benchmarking GPU Tensor Cores on General Matrix Multiplication Kernels through CUTLASS},
  journal = {Applied Sciences},
  year = {2023},
  volume = {13},
  number = {24},
  pages = {13022},
  doi = {10.3390/app132413022}
}
Kostenlos starten →