EUVIMEDEUVIMED entdecken
Crossref

In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics

Niki van Stein, Diederick Vermetten, Thomas Bäck

ACM Transactions on Evolutionary Learning and Optimization · 23.04.2025

DOI: 10.1145/3731567 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Evolutionary Learning and Optimization, online erschienen am 23.04.2025. Verfasst von Niki van Stein und 2 weiteren Autor:innen. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 31 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) have shown great potential in automatically generating and optimizing (meta)heuristics, making them valuable tools in heuristic optimization tasks. However, LLMs are generally inefficient when it comes to fine-tuning hyper-parameters of the generated algorithms, often requiring excessive queries that lead to high computational and financial costs. This paper presents a novel hybrid approach, LLaMEA-HPO, which integrates the open source LLaMEA (Large Language Model Evolutionary Algorithm) framework with a Hyper-Parameter Optimization (HPO) procedure in the loop. By offloading hyper-parameter tuning to an HPO procedure, the LLaMEA-HPO framework allows the LLM to focus on generating novel algorithmic structures, reducing the number of required LLM queries and improving the overall efficiency of the optimization process. We empirically validate the proposed hybrid framework on benchmark problems, including Online Bin Packing, Black-Box Optimization, and the Traveling Salesperson Problem. Our results demonstrate that LLaMEA-HPO achieves superior or comparable performance compared to existing LLM-driven frameworks while significantly reducing computational costs. This work highlights the importance of separating algorithmic innovation and structural code search from parameter tuning in LLM-driven code optimization and offers a scalable approach to improve the efficiency and effectiveness of LLM-based code generation.

Zitieren

APA

Stein, N. V., Vermetten, D., & Bäck, T. (2025). In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics. ACM Transactions on Evolutionary Learning and Optimization. https://doi.org/10.1145/3731567

Vancouver

Stein NV, Vermetten D, Bäck T. In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics. ACM Transactions on Evolutionary Learning and Optimization. 2025. doi:10.1145/3731567

BibTeX
@article{Stein2025,
  author = {Stein, Niki van and Vermetten, Diederick and Bäck, Thomas},
  title = {In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics},
  journal = {ACM Transactions on Evolutionary Learning and Optimization},
  year = {2025},
  doi = {10.1145/3731567}
}
Kostenlos starten →