FLARE: A Fine-Grained In-Memory Computing Heterogeneous Multi-Chiplet Accelerator for Large Language Models
IEEE Journal on Emerging and Selected Topics in Circuits and Systems · 2026-06 · 16 (2) · 500-513
DOI: 10.1109/jetcas.2026.3677596 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2026-06. Verfasst von Abhishek Tyagi und 5 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Gefördert von: Ministry of Education Academic Research Fund Tier 1 and Tier 2, National Research Foundation, Prime Minister’s Office, Singapore, under its Competitive Research Program.
Abstract (Originalsprache)
Deploying multi-billion-parameter Large Language Models (LLMs) on modern AI accelerators requires both low-latency communication and energy-efficient computation. Emerging heterogeneous in-memory computing (IMC) architectures offer a promising solution by co-locating compute and storage, thereby reducing data movement and power consumption. However, efficiently partitioning and distributing model workloads across cores remains a key open challenge in multi-chiplet, network-based accelerator systems for LLMs, as it directly impacts the workload’s performance and hardware utilization. We present a heterogeneous multi-chiplet accelerator architecture accompanied by a co-designed fine-grained hardware–software stack. This stack incorporates custom algorithms for LLM partitioning, workload mapping, and identification of hardware configurations that ensure efficient execution. In contrast to prior work, our methodology adopts a bottom-up perspective, addressing execution from the core level up to the chiplet-level coordination to enable scalability and flexible dataflows. Moreover, we assess and recommend hardware setups optimized for different classes of LLMs, offering practical guidance for selecting architectures best suited to specific models. Through cycle-accurate experimental evaluation, we demonstrate that our approach achieves up to $4\times $ higher token throughput and $30\times $ better energy efficiency compared to state-of-the-art H100 GPUs on the same workloads.
Förderung
- Ministry of Education Academic Research Fund Tier 1 and Tier 2
- National Research Foundation, Prime Minister’s Office, Singapore, under its Competitive Research Program
Zitieren
APA
Tyagi, A., Wang, Z., Harkishanka, A., Wong, H. G., Veeravalli, B., & Fong, X. (2026). FLARE: A Fine-Grained In-Memory Computing Heterogeneous Multi-Chiplet Accelerator for Large Language Models. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 16(2), 500-513. https://doi.org/10.1109/jetcas.2026.3677596
Vancouver
Tyagi A, Wang Z, Harkishanka A, Wong HG, Veeravalli B, Fong X. FLARE: A Fine-Grained In-Memory Computing Heterogeneous Multi-Chiplet Accelerator for Large Language Models. IEEE Journal on Emerging and Selected Topics in Circuits and Systems. 2026;16(2):500-513. doi:10.1109/jetcas.2026.3677596
BibTeX
@article{Tyagi2026,
author = {Tyagi, Abhishek and Wang, Zixuan and Harkishanka, Avadh and Wong, Hong Gao and Veeravalli, Bharadwaj and Fong, Xuanyao},
title = {FLARE: A Fine-Grained In-Memory Computing Heterogeneous Multi-Chiplet Accelerator for Large Language Models},
journal = {IEEE Journal on Emerging and Selected Topics in Circuits and Systems},
year = {2026},
volume = {16},
number = {2},
pages = {500--513},
doi = {10.1109/jetcas.2026.3677596}
}
EUVIMED