BlockSmith: Ultra-dense Block Transformation for Efficient Sparse Matrix Multiplication Using Tensor Cores
ACM Transactions on Architecture and Code Optimization · 30.09.2026 · 23 (3) · 1-24
DOI: 10.1145/3828547 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Architecture and Code Optimization, online erschienen am 24.07.2026. Verfasst von Yukang Dong und 6 weiteren Autor:innen. Die Arbeit nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China.
Abstract (Originalsprache)
Sparse-dense Matrix Multiplication (SpMM) is a fundamental operation in various fields. Recently, leveraging Tensor Cores (TCs) on GPUs to accelerate SpMM has attracted growing interest. Given the block-wise computation nature of TCs, existing TC-based SpMM methods typically apply preprocessing techniques, such as similarity-based row reordering, to cluster similar rows and improve data locality, with the goal of forming denser matrix blocks that better utilize TCs and enhance SpMM performance. However, these methods typically apply TCs only at the final computation stage, without fully accounting for the block-wise execution characteristic during the reordering process. As a result, the clustering outcomes often misalign with the actual computation pattern of TCs, limiting potential performance gains. To this end, we propose BlockSmith , an end-to-end framework for accelerating SpMM by leveraging TCs. The name BlockSmith reflects its core design principle: to forge high-density matrix blocks from sparse matrices. The key innovation lies in the Ultra-dense Block Transformation (UBT) approach, which significantly improves the data locality of sparse matrices by integrating a novel row-reordering algorithm and column compression technique, while also taking into account the block-wise characteristics of TCs. Moreover, we design a novel storage format to maximize memory access efficiency, together with an optimized GPU kernel that leverages low-level PTX instructions to fully exploit the capabilities of TCs. Extensive evaluations on modern GPUs, including RTX 4090 and RTX 5090, show that BlockSmith outperforms the state-of-the-art SpMM method Acc-SpMM, achieving average speedups of 1.89× and 4.35×, respectively, with the maximum speedup exceeding 100×. Moreover, for GNN inference tasks, BlockSmith-GCN achieves over 10× speedup compared with mainstream frameworks such as PyG and DGL. The source code of BlockSmith is publicly available at: https://github.com/CGCL-codes/BlockSmith.git .
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China
- Hubei Provincial Natural Science Foundation of China
Zitieren
APA
Dong, Y., Chen, H., Meng, X., Jiang, W., Yang, D., Xiang, E., & Jin, H. (2026). BlockSmith: Ultra-dense Block Transformation for Efficient Sparse Matrix Multiplication Using Tensor Cores. ACM Transactions on Architecture and Code Optimization, 23(3), 1-24. https://doi.org/10.1145/3828547
Vancouver
Dong Y, Chen H, Meng X, Jiang W, Yang D, Xiang E, et al. BlockSmith: Ultra-dense Block Transformation for Efficient Sparse Matrix Multiplication Using Tensor Cores. ACM Transactions on Architecture and Code Optimization. 2026;23(3):1-24. doi:10.1145/3828547
BibTeX
@article{Dong2026,
author = {Dong, Yukang and Chen, Hao and Meng, Xiangwenxin and Jiang, Wenbin and Yang, Daowen and Xiang, Enze and Jin, Hai},
title = {BlockSmith: Ultra-dense Block Transformation for Efficient Sparse Matrix Multiplication Using Tensor Cores},
journal = {ACM Transactions on Architecture and Code Optimization},
year = {2026},
volume = {23},
number = {3},
pages = {1--24},
doi = {10.1145/3828547}
}
EUVIMED