InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference
IEEE Computer Architecture Letters · 2026-01 · 25 (1) · 198-201
DOI: 10.1109/lca.2026.3693303 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Computer Architecture Letters, 2026-01. Verfasst von Kyeonghyeon Ryu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 6 Literaturangaben. Gefördert von: Institute of Information and Communications Technology Planning and Evaluation, National Research Foundation of Korea, Korea government.
Abstract (Originalsprache)
Intel Advanced Matrix Extensions (AMX) offers a theoretical peak of over 90 TFLOPS for LLM inference, yet practical efficiency often plateaus at 5–40% of its potential. Our analysis reveals that this performance gap stems from a fundamental architectural mismatch: bursty AMX memory accesses frequently saturate Line Fill Buffers (LFB), and naive multi-socket scaling triggers prohibitive synchronization overhead. To bridge this gap, we propose InfAMAX, an LLM inference framework built upon two key ideas. First, we introduce decoupled tile-pipelining and multi-level adaptive prefetching to maximize data reuse and mask LFB-induced latencies across the cache hierarchy. Second, we employ a NUMA-aware resource management strategy that enforces strict thread affinity and shape-aware partitioning to eliminate cross-socket cache coherence traffic. Experimental results show that InfAMAX improves end-to-end inference throughput by up to 2× (reducing decoding latency by 2×) over state-of-the-art libraries, providing a scalable and efficient foundation for high-performance LLM serving on modern CPUs.
Förderung
- Institute of Information and Communications Technology Planning and Evaluation
- National Research Foundation of Korea
- Korea government
Zitieren
APA
Ryu, K., Jeon, Y., Park, G., Kim, M., & Kim, D. (2026). InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference. IEEE Computer Architecture Letters, 25(1), 198-201. https://doi.org/10.1109/lca.2026.3693303
Vancouver
Ryu K, Jeon Y, Park G, Kim M, Kim D. InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference. IEEE Computer Architecture Letters. 2026;25(1):198-201. doi:10.1109/lca.2026.3693303
BibTeX
@article{Ryu2026,
author = {Ryu, Kyeonghyeon and Jeon, Yunhyeong and Park, Gyeongseo and Kim, Minho and Kim, Daehoon},
title = {InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference},
journal = {IEEE Computer Architecture Letters},
year = {2026},
volume = {25},
number = {1},
pages = {198--201},
doi = {10.1109/lca.2026.3693303}
}
EUVIMED