A 65-nm Compute-in-Read-Only-Memory Macro With Bidirectional Ternary Weight Storage and Sparsity-Aware Local Accumulation for 1.58-bit LLM Inference
IEEE Transactions on Circuits and Systems I: Regular Papers · 2026-11 · 73 (11) · 8057-8070
DOI: 10.1109/tcsi.2026.3721732 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems I: Regular Papers, 2026-11. Verfasst von Wenlun Zhang und 4 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Gefördert von: the Japan Science and Technology (JST) Core Research for Evolutional Science and Technology, JST Next-Generation Edge AI Semiconductor, JST Support for Pioneering Research Initiated by the Next Generation.
Abstract (Originalsprache)
Compute-in-Read-Only-Memory (CiROM) is a promising paradigm for weight reload-free edge inference. The recent emergence of BitNet, which represents weights in ternary values at an average precision of approximately 1.58 bits, opens a path toward single-chip LLM deployment by drastically reducing storage footprint. However, fully exploiting this opportunity requires addressing three key challenges: compact ternary weight storage, effective exploitation of high weight sparsity, and post-fabrication adaptability for downstream tasks. To this end, this paper presents a BitNet-oriented CiROM macro with three dedicated techniques. First, a Bidirectional ROM Array based on Configurable Source/Bit-Lines stores two ternary weights in each 1T ROM cell, significantly improving the weight storage density. Second, a Tri-Mode Local Accumulator is developed to support an accumulation-then-summation dataflow, where zero-weight operations are locally gated to translate BitNet sparsity into reduced switching activity and energy consumption. Third, a lightweight digital adapter co-designed with LoRA-based fine-tuning is introduced to provide post-fabrication downstream task adaptability with negligible hardware overhead. The proposed macro is fabricated in a 65nm CMOS process and achieves an ultra-high weight bit density of 4,479 Kb/mm2, corresponding to an $9.20\times $ improvement over prior digital CiROM designs. Silicon measurements demonstrate a peak energy efficiency of 33.7 TOPS/W and a peak compute density of 60.8 GOPS/mm2 over a 0.6-1.2 V supply range. Model-level evaluations further confirm that the proposed digital adapter consistently improves downstream performance across diverse tasks. These results demonstrate that the proposed macro extends CiROM toward efficient weight reload-free LLM acceleration, providing a promising path for compact and energy-efficient edge deployment.
Förderung
- the Japan Science and Technology (JST) Core Research for Evolutional Science and Technology
- JST Next-Generation Edge AI Semiconductor
- JST Support for Pioneering Research Initiated by the Next Generation
- the Japan Society for the Promotion of Science (JSPS) KAKENHI
- KIOXIA Encouragement Research Grant
Zitieren
APA
Zhang, W., Li, X., Ando, S., Xu, Z., & Yoshioka, K. (2026). A 65-nm Compute-in-Read-Only-Memory Macro With Bidirectional Ternary Weight Storage and Sparsity-Aware Local Accumulation for 1.58-bit LLM Inference. IEEE Transactions on Circuits and Systems I: Regular Papers, 73(11), 8057-8070. https://doi.org/10.1109/tcsi.2026.3721732
Vancouver
Zhang W, Li X, Ando S, Xu Z, Yoshioka K. A 65-nm Compute-in-Read-Only-Memory Macro With Bidirectional Ternary Weight Storage and Sparsity-Aware Local Accumulation for 1.58-bit LLM Inference. IEEE Transactions on Circuits and Systems I: Regular Papers. 2026;73(11):8057-8070. doi:10.1109/tcsi.2026.3721732
BibTeX
@article{Zhang2026,
author = {Zhang, Wenlun and Li, Xinyu and Ando, Shimpei and Xu, Zhen and Yoshioka, Kentaro},
title = {A 65-nm Compute-in-Read-Only-Memory Macro With Bidirectional Ternary Weight Storage and Sparsity-Aware Local Accumulation for 1.58-bit LLM Inference},
journal = {IEEE Transactions on Circuits and Systems I: Regular Papers},
year = {2026},
volume = {73},
number = {11},
pages = {8057--8070},
doi = {10.1109/tcsi.2026.3721732}
}
EUVIMED