HydraPIM: A Heterogeneous PIM Architecture for Efficient Attention in Long-Context LLMs
IEEE Transactions on Computers · 2026-09 · 75 (9) · 3383-3396
DOI: 10.1109/tc.2026.3710733 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Computers, 2026-09. Verfasst von Shixin Zhao und 9 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben.
Abstract (Originalsprache)
The growing demand for long-context LLM inference has exposed a critical bandwidth–capacity trade-off in memory systems, rendering single-tier PIM architectures ineffective. HBM-PIMs offer high bandwidth but limited capacity, while DIMM-PIMs provide scalability at the cost of lower bandwidth; neither satisfies the throughput-latency requirements of long-context serving workloads for LLMs. To address this, we propose HydraPIM , a heterogeneous multi-tier PIM architecture that decomposes attention computation across HBM-PIM and DIMM-PIM tiers by exploiting the inherent sparsity of attention. HydraPIM introduces HydraAttention , a tiled attention mechanism with hierarchical reduction, enabling efficient cross-tier execution through lightweight on-chip reduction units. To maintain load balance under dynamic access patterns of sparse attention, HydraPIM features an importance-aware KV migration mechanism that monitors token relevance and relocates hot tokens to high-bandwidth tiers at runtime. This software-hardware co-design helps improve the utilization of both bandwidth and capacity. Evaluations show that HydraPIM achieves 1.66 $\boldsymbol{\times}$ × higher throughput than HBM-based NPU-PIM and 1.96 $\boldsymbol{\times}$ × lower latency than DIMM-based NPU-PIM across diverse long-context workloads.
Zitieren
APA
Zhao, S., Liu, L., An, X., Zhou, Y., He, Y., Cheng, L., Wang, M., Li, X., Han, Y., & Wang, Y. (2026). HydraPIM: A Heterogeneous PIM Architecture for Efficient Attention in Long-Context LLMs. IEEE Transactions on Computers, 75(9), 3383-3396. https://doi.org/10.1109/tc.2026.3710733
Vancouver
Zhao S, Liu L, An X, Zhou Y, He Y, Cheng L, et al. HydraPIM: A Heterogeneous PIM Architecture for Efficient Attention in Long-Context LLMs. IEEE Transactions on Computers. 2026;75(9):3383-3396. doi:10.1109/tc.2026.3710733
BibTeX
@article{Zhao2026,
author = {Zhao, Shixin and Liu, Lian and An, Xiangwen and Zhou, Yutian and He, Yintao and Cheng, Long and Wang, Mengdi and Li, Xiaowei and Han, Yinhe and Wang, Ying},
title = {HydraPIM: A Heterogeneous PIM Architecture for Efficient Attention in Long-Context LLMs},
journal = {IEEE Transactions on Computers},
year = {2026},
volume = {75},
number = {9},
pages = {3383--3396},
doi = {10.1109/tc.2026.3710733}
}
EUVIMED