EUVIMEDEUVIMED entdecken
Crossref

Improving Carbon Emissions of Federated Large Language Model Inference through Classification of Task-Specificity

Geerd-Dietger Hoffmann, Verena Majuntke

ACM SIGEnergy Energy Informatics Review · 2024-12 · 4 (5) · 44-50

DOI: 10.1145/3727200.3727208 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM SIGEnergy Energy Informatics Review, online erschienen am 17.04.2025. Verfasst von Geerd-Dietger Hoffmann und 1 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 39 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The resource consumption of software and communication infrastructure is an increasing concern, particularly with the emergence of Large Language Models (LLMs). While energy consumption and carbon emission of LLMs during training have been a focus in research, the impact of LLM inference which scales with the number of requests, remains underexplored. In this paper we show that energy efficiency and carbon emission of LLM inference vary depending on the model and on the task category, e.g. math, programming, general knowledge, with which the model is prompted and that smaller specialized models can achieve comparable accuracy while using less resources. We analyze the differences across 8 open-source LLMs when processing prompts of different task categories. Our findings lead to a novel approach: Classifying prompts by using embeddings to route them to the most energy-efficient and least carbon-intensive LLM in a federation of LLMs while keeping accuracy high. We validate the effectiveness of our method through empirical measurements.

Zitieren

APA

Hoffmann, G., & Majuntke, V. (2024). Improving Carbon Emissions of Federated Large Language Model Inference through Classification of Task-Specificity. ACM SIGEnergy Energy Informatics Review, 4(5), 44-50. https://doi.org/10.1145/3727200.3727208

Vancouver

Hoffmann G, Majuntke V. Improving Carbon Emissions of Federated Large Language Model Inference through Classification of Task-Specificity. ACM SIGEnergy Energy Informatics Review. 2024;4(5):44-50. doi:10.1145/3727200.3727208

BibTeX
@article{Hoffmann2024,
  author = {Hoffmann, Geerd-Dietger and Majuntke, Verena},
  title = {Improving Carbon Emissions of Federated Large Language Model Inference through Classification of Task-Specificity},
  journal = {ACM SIGEnergy Energy Informatics Review},
  year = {2024},
  volume = {4},
  number = {5},
  pages = {44--50},
  doi = {10.1145/3727200.3727208}
}
Kostenlos starten →