EUVIMEDEUVIMED entdecken
Crossref

The Breakthrough Memory Solutions for Improved Performance on LLM Inference

Byeongho Kim, Sanghoon Cha, Sangsoo Park, Jieun Lee, Sukhan Lee, Shin-haeng Kang, Jinin So, Kyungsoo Kim, Jin Jung, Jong-Geon Lee, Sunjung Lee, Yoonah Paik und 11 weitere

IEEE Micro · 2024-05 · 44 (3) · 40-48

DOI: 10.1109/mm.2024.3375352 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Micro, 2024-05. Verfasst von Byeongho Kim und 22 weiteren Autor:innen. Die Arbeit wurde bisher 33-mal zitiert (Crossref) und nennt 10 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have changed our lives, but they require unprecedented computing resources—especially large memory capacity and high bandwidth to process weights. However, while the logic process was developing, the speed of development of the memory process could not keep up, causing problems that resulted in the performance of LLMs being hindered by memory. Samsung has introduced breakthrough processing-in-memory/processing-near-memory (PIM/PNM) solutions that enhance the main memory bandwidth. With the high bandwidth memory PIM-based GPU-cluster system and LPDDR5-PIM-based system, the performance of transformer-based LLMs improved by up to 1.9$ \times $× and 2.7$ \times $×, respectively. The Compute eXpress Link (CXL)-based PNM solution serves memory-centric computing systems by implementing logic inside the CXL memory controller. This results in a performance gain of more than 4.4$ \times $× with an energy reduction of about 53% with PNM. Furthermore, we provide PIM/PNM software stacks, including an AI compiler targeting the acceleration of AI models.

Zitieren

APA

Kim, B., Cha, S., Park, S., Lee, J., Lee, S., Kang, S., So, J., Kim, K., Jung, J., Lee, J., Lee, S., Paik, Y., Kim, H., Kim, J., Lee, W., Ro, Y., Cho, Y., Kim, J. H., Song, J., …, & Sohn, K. (2024). The Breakthrough Memory Solutions for Improved Performance on LLM Inference. IEEE Micro, 44(3), 40-48. https://doi.org/10.1109/mm.2024.3375352

Vancouver

Kim B, Cha S, Park S, Lee J, Lee S, Kang S, et al. The Breakthrough Memory Solutions for Improved Performance on LLM Inference. IEEE Micro. 2024;44(3):40-48. doi:10.1109/mm.2024.3375352

BibTeX
@article{Kim2024,
  author = {Kim, Byeongho and Cha, Sanghoon and Park, Sangsoo and Lee, Jieun and Lee, Sukhan and Kang, Shin-haeng and So, Jinin and Kim, Kyungsoo and Jung, Jin and Lee, Jong-Geon and Lee, Sunjung and Paik, Yoonah and Kim, Hyeonsu and Kim, Jin-Seong and Lee, Won-Jo and Ro, Yuhwan and Cho, YeonGon and Kim, Jin Hyun and Song, JoonHo and Yu, Jaehoon and Lee, Seungwon and Cho, Jeonghyeon and Sohn, Kyomin},
  title = {The Breakthrough Memory Solutions for Improved Performance on LLM Inference},
  journal = {IEEE Micro},
  year = {2024},
  volume = {44},
  number = {3},
  pages = {40--48},
  doi = {10.1109/mm.2024.3375352}
}
Kostenlos starten →