EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Analyzing the Impact of Kernel Fusion on GPU Tensor Operation Performance: A Systematic Performance Study

Matija Dodović, Milica Veselinović, Marko Mišić

Electronics · 02.03.2026 · 15 (5) · 1034

DOI: 10.3390/electronics15051034 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 02.03.2026. Verfasst von Matija Dodović und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science, Technological Development and Innovation of the Republic of Serbia.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large numbers of small tensor kernels are executed by GPUs in modern deep learning frameworks, where total performance is frequently constrained by memory bandwidth and kernel launch overheads. Systems such as TensorFlow XLA, PyTorch JIT, and cuDNN often use kernel fusion, which is defined as combining many tensor operations into a single GPU kernel, to reduce intermediate memory transfers and boost efficiency. Nevertheless, it is difficult to measure the true performance impact of fusion on both isolated tensor operations and end-to-end model execution. An experimental investigation of kernel fusion on three different NVIDIA GPUs is presented in this work. For four sample tensor operations: element-wise addition, fused multiply–add, linear transformation with ReLU activation, and map-reduce, we build fused and unfused CUDA kernels using FP32, FP16, and mixed-precision arithmetics. We measure execution time, speedup, and effective memory bandwidth across a range of input sizes. For memory-bound and activation-heavy workloads, fusion yields consistent speedups between 1.5× and 3.13×, particularly for small and medium inputs where kernel launch overhead is significant. For operations dominated by atomic updates, the benefit is limited to between 1.01× and 1.44×. When the reduction strategy is reformulated using block-level shared-memory aggregation, kernel fusion becomes effective again, achieving speedups of up to 2× by eliminating global synchronization bottlenecks. We further evaluate the effect of fusion on image classification models using PyTorch 2.10.0 JIT, achieving 1.54× to 1.83× faster inference. Our results provide practical guidelines on when kernel fusion is most effective.

Förderung

  • Ministry of Science, Technological Development and Innovation of the Republic of Serbia

Zitieren

APA

Dodović, M., Veselinović, M., & Mišić, M. (2026). Analyzing the Impact of Kernel Fusion on GPU Tensor Operation Performance: A Systematic Performance Study. Electronics, 15(5), 1034. https://doi.org/10.3390/electronics15051034

Vancouver

Dodović M, Veselinović M, Mišić M. Analyzing the Impact of Kernel Fusion on GPU Tensor Operation Performance: A Systematic Performance Study. Electronics. 2026;15(5):1034. doi:10.3390/electronics15051034

BibTeX
@article{Dodovi2026,
  author = {Dodović, Matija and Veselinović, Milica and Mišić, Marko},
  title = {Analyzing the Impact of Kernel Fusion on GPU Tensor Operation Performance: A Systematic Performance Study},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {5},
  pages = {1034},
  doi = {10.3390/electronics15051034}
}
Kostenlos starten →