EUVIMEDEUVIMED entdecken
Crossref

Energy Efficient or Exhaustive? Benchmarking Power Consumption of LLM Inference Engines

Chenxu Niu, Wei Zhang, Yongjian Zhao, Yong Chen

ACM SIGEnergy Energy Informatics Review · 2025-07 · 5 (2) · 56-62

DOI: 10.1145/3757892.3757900 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM SIGEnergy Energy Informatics Review, online erschienen am 06.08.2025. Verfasst von Chenxu Niu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 31 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) have remarkable advancements in recent years and have revolutionized the field of natural language processing. To reduce latency and improve inference throughput, many inference engines have been proposed such as vLLM, TensorRT-LLM, and DeepSpeed. However, there is no comprehensive analysis on the power consumption and energy efficiency of these inference engines. In this paper, we benchmark the power consumption of LLM inference engines on one single GPU node with 2 H100 GPUs and provide a fine-grained analysis by decomposing the inference lifecycle into two stages: the setup stage including engine initialization and model loading; and the token generation stage. For each stage, we further measure power consumption across key system components, including GPU, CPU, and DRAM. This breakdown analysis allows us to identify energy bottlenecks of inference lifecycle and gain deeper insights into the energy efficiency of modern inference engines.

Zitieren

APA

Niu, C., Zhang, W., Zhao, Y., & Chen, Y. (2025). Energy Efficient or Exhaustive? Benchmarking Power Consumption of LLM Inference Engines. ACM SIGEnergy Energy Informatics Review, 5(2), 56-62. https://doi.org/10.1145/3757892.3757900

Vancouver

Niu C, Zhang W, Zhao Y, Chen Y. Energy Efficient or Exhaustive? Benchmarking Power Consumption of LLM Inference Engines. ACM SIGEnergy Energy Informatics Review. 2025;5(2):56-62. doi:10.1145/3757892.3757900

BibTeX
@article{Niu2025,
  author = {Niu, Chenxu and Zhang, Wei and Zhao, Yongjian and Chen, Yong},
  title = {Energy Efficient or Exhaustive? Benchmarking Power Consumption of LLM Inference Engines},
  journal = {ACM SIGEnergy Energy Informatics Review},
  year = {2025},
  volume = {5},
  number = {2},
  pages = {56--62},
  doi = {10.1145/3757892.3757900}
}
Kostenlos starten →