EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Insights into resource utilization of code small language models serving with runtime engines and execution providers

Francisco Durán, Matias Martinez, Patricia Lago, Silverio Martínez-Fernández

Journal of Systems and Software · 2025-12 · 230 · 112574

DOI: 10.1016/j.jss.2025.112574 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Systems and Software, 2025-12. Verfasst von Francisco Durán und 3 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 97 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Commission, Gobierno de España Ministerio de Ciencia, Innovación y Universidades.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The rapid growth of language models, particularly in code generation, requires substantial computational resources, raising concerns about energy consumption and environmental impact. Optimizing language models inference resource utilization is crucial, and Small Language Models (SLMs) offer a promising solution to reduce resource demands. Our goal is to analyze the impact of deep learning serving configurations, defined as combinations of runtime engines and execution providers, on resource utilization, in terms of energy consumption, execution time, and computing-resource utilization from the point of view of software engineers conducting inference in the context of code generation SLMs. We conducted a technology-oriented, multi-stage experimental pipeline using twelve code generation SLMs to investigate energy consumption, execution time, and computing-resource utilization across the configurations. Significant differences emerged across configurations. CUDA execution provider configurations outperformed CPU execution provider configurations in both energy consumption and execution time. Among the configurations, TORCH paired with CUDA demonstrated the greatest energy efficiency, achieving energy savings from 37.99% up to 89.16% compared to other serving configurations. Similarly, optimized runtime engines like ONNX with the CPU execution provider achieved from 8.98% up to 72.04% energy savings within CPU-based configurations. Also, TORCH paired with CUDA exhibited efficient computing-resource utilization. Serving configuration choice significantly impacts resource utilization. While further research is needed, we recommend the above configurations best suited to software engineers'requirements for enhancing serving resource utilization efficiency.

Förderung

  • European Commission
  • Gobierno de España Ministerio de Ciencia, Innovación y Universidades

Zitieren

APA

Durán, F., Martinez, M., Lago, P., & Martínez-Fernández, S. (2025). Insights into resource utilization of code small language models serving with runtime engines and execution providers. Journal of Systems and Software, 230, 112574. https://doi.org/10.1016/j.jss.2025.112574

Vancouver

Durán F, Martinez M, Lago P, Martínez-Fernández S. Insights into resource utilization of code small language models serving with runtime engines and execution providers. Journal of Systems and Software. 2025;230:112574. doi:10.1016/j.jss.2025.112574

BibTeX
@article{Durn2025,
  author = {Durán, Francisco and Martinez, Matias and Lago, Patricia and Martínez-Fernández, Silverio},
  title = {Insights into resource utilization of code small language models serving with runtime engines and execution providers},
  journal = {Journal of Systems and Software},
  year = {2025},
  volume = {230},
  pages = {112574},
  doi = {10.1016/j.jss.2025.112574}
}
Kostenlos starten →