EUVIMEDSkopri EUVIMED
Crossref

HyPIM: LLM Acceleration with A Hybrid ReRAM/SRAM 3D-PIM Architecture

Xiaolong Hu, Chubo Liu, Yan Ding, Keqin Li, Kenli Li

ACM Transactions on Embedded Computing Systems · 2026-06-10

DOI: 10.1145/3820366 ↗

At a glance

Journal article in ACM Transactions on Embedded Computing Systems, published online on 2026-06-10. Written by Xiaolong Hu and 4 co-authors. 54 references.

AstrattDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Transformer-based Large Language Models (LLMs) have made significant progress in the fields of computer vision and natural language processing. The use of LLMs in embedded and real-time systems to provide more powerful intelligence capabilities is also an inevitable trend. However, the traditional von Neumann architecture encounters great challenges with transformer-based LLMs due to huge data movement overheads. Processing-in-Memory (PIM) emerges as a great potential solution by significantly closing the gap between computation and storage. Nevertheless, transformer-based LLMs usually consist of multiple transformer blocks, and the operations within each transformer block vary significantly on computation and memory access characteristics. Existing PIM designs that rely solely on SRAM, DRAM, or ReRAM cannot adapt well to transformer-based LLMs. To address these challenges, we introduce HyPIM, a 3D hybrid PIM architecture that integrates SRAM and ReRAM slices to process through their cooperation. However, HyPIM does not change the scenario where the inference latency of the attention mechanism continues to dominate the overall inference latency in the transformer-based LLMs when dealing with long sequences of data. Exploiting these insights, we then design a hardware-friendly early termination approximation algorithm and a sparse matrix multiplication strategy for the HyPIM architecture, reducing the overhead through a software and hardware co-design. Compared with the state-of-the-art Newton, TransPIM, HAIMA, HARDSEA, and H3DAtten architectures, the proposed HyPIM architecture achieves up to a 1.27×–1.67× latency improvement.

Cite

APA

Hu, X., Liu, C., Ding, Y., Li, K., & Li, K. (2026). HyPIM: LLM Acceleration with A Hybrid ReRAM/SRAM 3D-PIM Architecture. ACM Transactions on Embedded Computing Systems. https://doi.org/10.1145/3820366

Vancouver

Hu X, Liu C, Ding Y, Li K, Li K. HyPIM: LLM Acceleration with A Hybrid ReRAM/SRAM 3D-PIM Architecture. ACM Transactions on Embedded Computing Systems. 2026. doi:10.1145/3820366

BibTeX
@article{Hu2026,
  author = {Hu, Xiaolong and Liu, Chubo and Ding, Yan and Li, Keqin and Li, Kenli},
  title = {HyPIM: LLM Acceleration with A Hybrid ReRAM/SRAM 3D-PIM Architecture},
  journal = {ACM Transactions on Embedded Computing Systems},
  year = {2026},
  doi = {10.1145/3820366}
}
Start for free →