Unleashing Triton on CPUs: Compilation and Runtime Co-Optimization for Scalable Vector Architectures
Computers · 2026-06-25 · 15 (7) · 406
DOI: 10.3390/computers15070406 ↗At a glance
Journal article in Computers, published online on 2026-06-25. Written by Jianan Li and 2 co-authors. 30 references. The full text is freely available (open access, CC BY 4.0). Funded by: National Key Research and Development Program of China, Henan Science and Technology Department.
Abstract (original language)
While the Triton compiler has revolutionized GPU kernel development, its deployment on general-purpose CPUs struggles to fully utilize the underlying hardware capabilities. This is primarily due to the semantic gap between Triton’s SPMD execution model and CPU vector architectures, which leads to suboptimal utilization of vector units during complex memory accesses. In this paper, we present a comprehensive compilation and runtime co-optimization framework for Triton-CPU, specifically targeting Vector Length Agnostic architectures (VLA) like ARM SVE. At the compiler level, we propose a novel semantic reconstruction and explicit base-offset decoupling strategy, enabling native VLA gather/scatter generation and eliminating scalar loop overheads. At the runtime level, we introduce a Machine Learning-driven thread scheduling model to optimally orchestrate the synergy between Thread-Level Parallelism and Vector-Level Parallelism. Extensive evaluations on an ARM-based multi-core processor demonstrate that our framework achieves up to a 2.0× throughput improvement for compute-bound GEMM operators (peaking at 346 GFLOPS), notably outperforming the hand-optimized OpenBLAS library by up to 1.54× at small-to-medium scales. Additionally, it delivers a 1.7× speedup for element-wise workloads. Furthermore, our optimizations saturate memory bandwidth (up to 55 GB/s) for memory-bound operators with zero compilation bloat, establishing a robust, high-performance foundation for deploying deep learning models on general-purpose CPUs.
Funding
- National Key Research and Development Program of China
- Henan Science and Technology Department
Cite
APA
Li, J., Chai, X., & Gao, W. (2026). Unleashing Triton on CPUs: Compilation and Runtime Co-Optimization for Scalable Vector Architectures. Computers, 15(7), 406. https://doi.org/10.3390/computers15070406
Vancouver
Li J, Chai X, Gao W. Unleashing Triton on CPUs: Compilation and Runtime Co-Optimization for Scalable Vector Architectures. Computers. 2026;15(7):406. doi:10.3390/computers15070406
BibTeX
@article{Li2026,
author = {Li, Jianan and Chai, Xiaonan and Gao, Wei},
title = {Unleashing Triton on CPUs: Compilation and Runtime Co-Optimization for Scalable Vector Architectures},
journal = {Computers},
year = {2026},
volume = {15},
number = {7},
pages = {406},
doi = {10.3390/computers15070406}
}
EUVIMED