EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

PyAO: PyTorch-Based Memory-Efficient LLM Training on Ethernet-Interconnected Clusters

Daemin Kim, Hyorim Kim, Juncheol Ahn, Sejin Park iD

Sensors · 07.04.2026 · 26 (7) · 2269

DOI: 10.3390/s26072269 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 07.04.2026. Verfasst von Daemin Kim und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Engineering, Keimyung University, Daegu 42601, Republic of Korea. Die Arbeit nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Daegu Digital Innovation Promotion Agency(DIP), Daegu Digital Innovation Promotion Agency (DIP) OF FUNDER.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As large language models (LLMs) pursue higher accuracy, their model sizes have surged, substantially increasing GPU memory consumption. Prior work mitigates this issue by distributing the memory burden across multiple GPUs. However, on clusters interconnected via Ethernet, the resulting computational intensity is insufficient to hide the significant network latency. Achieving a favorable compute-to-communication ratio is further constrained by the memory required to cache the massive activations generated during the forward pass. PyAO, proposed in this paper, effectively offloads activations, selects offloading strategies based on their offloading efficiency, and minimizes data-movement bottlenecks, thereby enabling larger micro-batch sizes. In Ethernet-interconnected cluster environments, experiments on popular models—including OPT-1.3B, GPT-0.8B, and Llama-1.2B—demonstrate that PyAO reduces peak GPU memory by up to 1.94× at the same micro-batch size, enables up to 2.5× larger batch sizes, and accelerates training by up to 3.63× relative to the baseline.

Schlagwörter

Stichwörter der Autor:innen

activation offloadingdistributed traininglow-bandwidth networkmemory-efficient training

Autor:innen und Institutionen

  • Department of Computer Engineering, Keimyung University, Daegu 42601, Republic of Korea.

ORCID-Profile: Sejin Park

Förderung

  • Daegu Digital Innovation Promotion Agency(DIP)
  • Daegu Digital Innovation Promotion Agency (DIP) OF FUNDER

Zitieren

APA

Kim, D., Kim, H., Ahn, J., & Park, S. (2026). PyAO: PyTorch-Based Memory-Efficient LLM Training on Ethernet-Interconnected Clusters. Sensors, 26(7), 2269. https://doi.org/10.3390/s26072269

Vancouver

Kim D, Kim H, Ahn J, Park S. PyAO: PyTorch-Based Memory-Efficient LLM Training on Ethernet-Interconnected Clusters. Sensors. 2026;26(7):2269. doi:10.3390/s26072269

BibTeX
@article{Kim2026,
  author = {Kim, Daemin and Kim, Hyorim and Ahn, Juncheol and Park, Sejin},
  title = {PyAO: PyTorch-Based Memory-Efficient LLM Training on Ethernet-Interconnected Clusters},
  journal = {Sensors},
  year = {2026},
  volume = {26},
  number = {7},
  pages = {2269},
  doi = {10.3390/s26072269}
}
Kostenlos starten →