EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-SA 4.0

User-driven Online Kernel Fusion for SYCL

Víctor Pérez, Lukas Sommer, Victor Lomüller, Kumudha Narasimhan, Mehdi Goli

ACM Transactions on Architecture and Code Optimization · 30.06.2023 · 20 (2) · 1-25

DOI: 10.1145/3571284 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Architecture and Code Optimization, 2023-03. Verfasst von Víctor Pérez und 4 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Heterogeneous programming models are becoming increasingly popular to support the ever-evolving hardware architectures, especially for new and emerging specialized accelerators optimizing specific tasks. While such programs provide performance portability of the existing applications across various heterogeneous architectures to some extent, short-running device kernels can affect an application performance due to overheads of data transfer, synchronization, and kernel launch. While in applications with one or two short-running kernels the overhead can be negligible, it can be noticeable when these short-running kernels dominate the overall number of kernels in an application, as it is the case in graph-based neural network models, where there are several small memory-bound nodes alongside few large compute-bound nodes. To reduce the overhead, combining several kernels into a single, more optimized kernel is an active area of research. However, this task can be time-consuming and error-prone given the huge set of potential combinations. This can push programmers to seek a tradeoff between (a) task-specific kernels with low overhead but hard to maintain and (b) smaller modular kernels with higher overhead but easier to maintain. While there are DSL-based approaches, such as those provided for machine learning frameworks, which offer the possibility of such a fusion, they are limited to a particular domain and exploit specific knowledge of that domain and, as a consequence, are hard to port elsewhere. This study explores the feasibility of a user-driven kernel fusion through an extension to the SYCL API to address the automation of kernel fusion. The proposed solution requires programmers to define the subgraph regions that are potentially suitable for fusion without any modification to the kernel code or the function signature. We evaluate the performance benefit of our approach on common neural networks and study the performance improvement in detail.

Zitieren

APA

Pérez, V., Sommer, L., Lomüller, V., Narasimhan, K., & Goli, M. (2023). User-driven Online Kernel Fusion for SYCL. ACM Transactions on Architecture and Code Optimization, 20(2), 1-25. https://doi.org/10.1145/3571284

Vancouver

Pérez V, Sommer L, Lomüller V, Narasimhan K, Goli M. User-driven Online Kernel Fusion for SYCL. ACM Transactions on Architecture and Code Optimization. 2023;20(2):1-25. doi:10.1145/3571284

BibTeX
@article{Prez2023,
  author = {Pérez, Víctor and Sommer, Lukas and Lomüller, Victor and Narasimhan, Kumudha and Goli, Mehdi},
  title = {User-driven Online Kernel Fusion for SYCL},
  journal = {ACM Transactions on Architecture and Code Optimization},
  year = {2023},
  volume = {20},
  number = {2},
  pages = {1--25},
  doi = {10.1145/3571284}
}
Kostenlos starten →