EUVIMEDEUVIMED entdecken
Crossref

Towards Scalable Storage Architectures for GPU Clusters Running Large Language Models

Ali Sedaghatgoo, Reza Salkhordeh, André Brinkmann, Hossein Asadi

Proceedings of the ACM on Measurement and Analysis of Computing Systems · 26.03.2026 · 10 (1) · 1-26

DOI: 10.1145/3788106 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Measurement and Analysis of Computing Systems, online erschienen am 26.03.2026. Verfasst von Ali Sedaghatgoo und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 61 Literaturangaben. Gefördert von: Iran National Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In modern multi-accelerator nodes, GPU throughput is increasingly constrained by storage and I/O bottlenecks, leaving accelerators idle as data transfer is restricted by software. In this study, we focus on single-node, multi-GPU systems with small-to-medium-scale models and present a systematic, phase-aware evaluation of datapath performance for Large Language Models (LLMs), covering technologies such as in-kernel libaio, hybrid user-kernel io_uring, user-space NVMe via the Storage Performance Development Kit (SPDK), and GPUDirect Storage (GDS). These approaches are evaluated across various storage media including SATA Solid State Drives (SSDs), NVMe SSDs, Optane NVMe, and Optane Persistent Memory (PMem). Leveraging an automated evaluation framework, we explore over 25,000 configurations, measuring throughput, latency, I/O per second (IOPS), and CPU cost. Our study offers LLM storage scenarios in both standardized benchmarks and real-world production traces, ensuring that our workload models accurately reflect the I/O demands across pre-training, fine-tuning, and inference. We find that for inference, io_uring achieves the lowest latency and competitive IOPS for small random I/O on NVMe. In contrast, SPDK is limited to raw block-device evaluation due to its lack of POSIX file-system support. For pre-training and fine-tuning, workloads are dominated by coarse-grained sequential reads and writes, where GDS excels in reducing load times and host CPU usage. Among CPU-mediated datapaths, CPU efficiency—measured as GB/s per core—emerges as the key differentiator. Taken together, these results yield actionable design guidelines: align the choice of datapath with the LLM pipeline phase. Use io_uring for inference to optimize data transfer efficiency and minimize latency, and leverage GDS for pre-training and fine-tuning to improve throughput per core, thereby narrowing the storage-to-compute gap in GPU LLM clusters.

Förderung

  • Iran National Science Foundation

Zitieren

APA

Sedaghatgoo, A., Salkhordeh, R., Brinkmann, A., & Asadi, H. (2026). Towards Scalable Storage Architectures for GPU Clusters Running Large Language Models. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 10(1), 1-26. https://doi.org/10.1145/3788106

Vancouver

Sedaghatgoo A, Salkhordeh R, Brinkmann A, Asadi H. Towards Scalable Storage Architectures for GPU Clusters Running Large Language Models. Proceedings of the ACM on Measurement and Analysis of Computing Systems. 2026;10(1):1-26. doi:10.1145/3788106

BibTeX
@article{Sedaghatgoo2026,
  author = {Sedaghatgoo, Ali and Salkhordeh, Reza and Brinkmann, André and Asadi, Hossein},
  title = {Towards Scalable Storage Architectures for GPU Clusters Running Large Language Models},
  journal = {Proceedings of the ACM on Measurement and Analysis of Computing Systems},
  year = {2026},
  volume = {10},
  number = {1},
  pages = {1--26},
  doi = {10.1145/3788106}
}
Kostenlos starten →