EUVIMEDEUVIMED entdecken
Crossref

RSR-core: A High-Performance Engine for Low-Bit Matrix-Vector Multiplication

Mohsen Dehghankar, Abolfazl Asudeh

Proceedings of the VLDB Endowment · 01.08.2026 · 19 (12) · 4726-4729

DOI: 10.14778/3827998.3828107 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 14.09.2026. Verfasst von Mohsen Dehghankar und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 12 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Matrix-vector multiplication is a fundamental building block in neural networks, vector databases, and large language models, particularly during inference. As a result, efficient matrix-vector multiplication engines directly translate into more efficient inference. Recent work has explored low-bit quantization of model weights, where matrices are represented using binary (1-bit) or ternary (1.58-bit) values while activation kept in higher precision. These representations enable efficient hardware-level computation. In parallel, algorithms such as Redundant Segment Reduction (RSR) provide theoretical guarantees for accelerating low-bit matrix-vector multiplication. However, existing implementations operate at the application level and cannot be efficiently integrated into hardware kernels, limiting practical performance. To bridge this gap, we present RSR-core, a high-performance engine that implements the RSR algorithm as optimized low-level kernels for both CPU and CUDA environments. RSR-core supports efficient matrix-vector multiplication for binary and ternary weight matrices and general vectors while enabling practical deployment of RSR algorithm in real inference pipelines. RSR-core is provided as a production-ready engine with HuggingFace integration for preprocessing low-bit models and running accelerated inference. Experimental results demonstrate significant performance improvements over baseline HuggingFace PyTorch multiplication, achieving up to 62× speedup on CPU and up to 1.9× speedup for token generation on CUDA for popular ternary LLMs. The source code is publicly available at RSR-core repository 1 .

Zitieren

APA

Dehghankar, M., & Asudeh, A. (2026). RSR-core: A High-Performance Engine for Low-Bit Matrix-Vector Multiplication. Proceedings of the VLDB Endowment, 19(12), 4726-4729. https://doi.org/10.14778/3827998.3828107

Vancouver

Dehghankar M, Asudeh A. RSR-core: A High-Performance Engine for Low-Bit Matrix-Vector Multiplication. Proceedings of the VLDB Endowment. 2026;19(12):4726-4729. doi:10.14778/3827998.3828107

BibTeX
@article{Dehghankar2026,
  author = {Dehghankar, Mohsen and Asudeh, Abolfazl},
  title = {RSR-core: A High-Performance Engine for Low-Bit Matrix-Vector Multiplication},
  journal = {Proceedings of the VLDB Endowment},
  year = {2026},
  volume = {19},
  number = {12},
  pages = {4726--4729},
  doi = {10.14778/3827998.3828107}
}
Kostenlos starten →