EUVIMEDEUVIMED entdecken
Crossref

InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference

Kyeonghyeon Ryu, Yunhyeong Jeon, Gyeongseo Park, Minho Kim, Daehoon Kim

IEEE Computer Architecture Letters · 2026-01 · 25 (1) · 198-201

DOI: 10.1109/lca.2026.3693303 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Computer Architecture Letters, 2026-01. Verfasst von Kyeonghyeon Ryu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 6 Literaturangaben. Gefördert von: Institute of Information and Communications Technology Planning and Evaluation, National Research Foundation of Korea, Korea government.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Intel Advanced Matrix Extensions (AMX) offers a theoretical peak of over 90 TFLOPS for LLM inference, yet practical efficiency often plateaus at 5–40% of its potential. Our analysis reveals that this performance gap stems from a fundamental architectural mismatch: bursty AMX memory accesses frequently saturate Line Fill Buffers (LFB), and naive multi-socket scaling triggers prohibitive synchronization overhead. To bridge this gap, we propose InfAMAX, an LLM inference framework built upon two key ideas. First, we introduce decoupled tile-pipelining and multi-level adaptive prefetching to maximize data reuse and mask LFB-induced latencies across the cache hierarchy. Second, we employ a NUMA-aware resource management strategy that enforces strict thread affinity and shape-aware partitioning to eliminate cross-socket cache coherence traffic. Experimental results show that InfAMAX improves end-to-end inference throughput by up to 2× (reducing decoding latency by 2×) over state-of-the-art libraries, providing a scalable and efficient foundation for high-performance LLM serving on modern CPUs.

Förderung

  • Institute of Information and Communications Technology Planning and Evaluation
  • National Research Foundation of Korea
  • Korea government

Zitieren

APA

Ryu, K., Jeon, Y., Park, G., Kim, M., & Kim, D. (2026). InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference. IEEE Computer Architecture Letters, 25(1), 198-201. https://doi.org/10.1109/lca.2026.3693303

Vancouver

Ryu K, Jeon Y, Park G, Kim M, Kim D. InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference. IEEE Computer Architecture Letters. 2026;25(1):198-201. doi:10.1109/lca.2026.3693303

BibTeX
@article{Ryu2026,
  author = {Ryu, Kyeonghyeon and Jeon, Yunhyeong and Park, Gyeongseo and Kim, Minho and Kim, Daehoon},
  title = {InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference},
  journal = {IEEE Computer Architecture Letters},
  year = {2026},
  volume = {25},
  number = {1},
  pages = {198--201},
  doi = {10.1109/lca.2026.3693303}
}
Kostenlos starten →