EUVIMEDEUVIMED entdecken
Crossref

FLARE: A Fine-Grained In-Memory Computing Heterogeneous Multi-Chiplet Accelerator for Large Language Models

Abhishek Tyagi, Zixuan Wang, Avadh Harkishanka, Hong Gao Wong, Bharadwaj Veeravalli, Xuanyao Fong

IEEE Journal on Emerging and Selected Topics in Circuits and Systems · 2026-06 · 16 (2) · 500-513

DOI: 10.1109/jetcas.2026.3677596 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2026-06. Verfasst von Abhishek Tyagi und 5 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Gefördert von: Ministry of Education Academic Research Fund Tier 1 and Tier 2, National Research Foundation, Prime Minister’s Office, Singapore, under its Competitive Research Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deploying multi-billion-parameter Large Language Models (LLMs) on modern AI accelerators requires both low-latency communication and energy-efficient computation. Emerging heterogeneous in-memory computing (IMC) architectures offer a promising solution by co-locating compute and storage, thereby reducing data movement and power consumption. However, efficiently partitioning and distributing model workloads across cores remains a key open challenge in multi-chiplet, network-based accelerator systems for LLMs, as it directly impacts the workload’s performance and hardware utilization. We present a heterogeneous multi-chiplet accelerator architecture accompanied by a co-designed fine-grained hardware–software stack. This stack incorporates custom algorithms for LLM partitioning, workload mapping, and identification of hardware configurations that ensure efficient execution. In contrast to prior work, our methodology adopts a bottom-up perspective, addressing execution from the core level up to the chiplet-level coordination to enable scalability and flexible dataflows. Moreover, we assess and recommend hardware setups optimized for different classes of LLMs, offering practical guidance for selecting architectures best suited to specific models. Through cycle-accurate experimental evaluation, we demonstrate that our approach achieves up to $4\times $ higher token throughput and $30\times $ better energy efficiency compared to state-of-the-art H100 GPUs on the same workloads.

Förderung

  • Ministry of Education Academic Research Fund Tier 1 and Tier 2
  • National Research Foundation, Prime Minister’s Office, Singapore, under its Competitive Research Program

Zitieren

APA

Tyagi, A., Wang, Z., Harkishanka, A., Wong, H. G., Veeravalli, B., & Fong, X. (2026). FLARE: A Fine-Grained In-Memory Computing Heterogeneous Multi-Chiplet Accelerator for Large Language Models. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 16(2), 500-513. https://doi.org/10.1109/jetcas.2026.3677596

Vancouver

Tyagi A, Wang Z, Harkishanka A, Wong HG, Veeravalli B, Fong X. FLARE: A Fine-Grained In-Memory Computing Heterogeneous Multi-Chiplet Accelerator for Large Language Models. IEEE Journal on Emerging and Selected Topics in Circuits and Systems. 2026;16(2):500-513. doi:10.1109/jetcas.2026.3677596

BibTeX
@article{Tyagi2026,
  author = {Tyagi, Abhishek and Wang, Zixuan and Harkishanka, Avadh and Wong, Hong Gao and Veeravalli, Bharadwaj and Fong, Xuanyao},
  title = {FLARE: A Fine-Grained In-Memory Computing Heterogeneous Multi-Chiplet Accelerator for Large Language Models},
  journal = {IEEE Journal on Emerging and Selected Topics in Circuits and Systems},
  year = {2026},
  volume = {16},
  number = {2},
  pages = {500--513},
  doi = {10.1109/jetcas.2026.3677596}
}
Kostenlos starten →