Analyzing the Impact of Kernel Fusion on GPU Tensor Operation Performance: A Systematic Performance Study
Electronics · 02.03.2026 · 15 (5) · 1034
DOI: 10.3390/electronics15051034 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 02.03.2026. Verfasst von Matija Dodović und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science, Technological Development and Innovation of the Republic of Serbia.
Abstract (Originalsprache)
Large numbers of small tensor kernels are executed by GPUs in modern deep learning frameworks, where total performance is frequently constrained by memory bandwidth and kernel launch overheads. Systems such as TensorFlow XLA, PyTorch JIT, and cuDNN often use kernel fusion, which is defined as combining many tensor operations into a single GPU kernel, to reduce intermediate memory transfers and boost efficiency. Nevertheless, it is difficult to measure the true performance impact of fusion on both isolated tensor operations and end-to-end model execution. An experimental investigation of kernel fusion on three different NVIDIA GPUs is presented in this work. For four sample tensor operations: element-wise addition, fused multiply–add, linear transformation with ReLU activation, and map-reduce, we build fused and unfused CUDA kernels using FP32, FP16, and mixed-precision arithmetics. We measure execution time, speedup, and effective memory bandwidth across a range of input sizes. For memory-bound and activation-heavy workloads, fusion yields consistent speedups between 1.5× and 3.13×, particularly for small and medium inputs where kernel launch overhead is significant. For operations dominated by atomic updates, the benefit is limited to between 1.01× and 1.44×. When the reduction strategy is reformulated using block-level shared-memory aggregation, kernel fusion becomes effective again, achieving speedups of up to 2× by eliminating global synchronization bottlenecks. We further evaluate the effect of fusion on image classification models using PyTorch 2.10.0 JIT, achieving 1.54× to 1.83× faster inference. Our results provide practical guidelines on when kernel fusion is most effective.
Förderung
- Ministry of Science, Technological Development and Innovation of the Republic of Serbia
Zitieren
APA
Dodović, M., Veselinović, M., & Mišić, M. (2026). Analyzing the Impact of Kernel Fusion on GPU Tensor Operation Performance: A Systematic Performance Study. Electronics, 15(5), 1034. https://doi.org/10.3390/electronics15051034
Vancouver
Dodović M, Veselinović M, Mišić M. Analyzing the Impact of Kernel Fusion on GPU Tensor Operation Performance: A Systematic Performance Study. Electronics. 2026;15(5):1034. doi:10.3390/electronics15051034
BibTeX
@article{Dodovi2026,
author = {Dodović, Matija and Veselinović, Milica and Mišić, Marko},
title = {Analyzing the Impact of Kernel Fusion on GPU Tensor Operation Performance: A Systematic Performance Study},
journal = {Electronics},
year = {2026},
volume = {15},
number = {5},
pages = {1034},
doi = {10.3390/electronics15051034}
}
EUVIMED