In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics
ACM Transactions on Evolutionary Learning and Optimization · 23.04.2025
DOI: 10.1145/3731567 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Evolutionary Learning and Optimization, online erschienen am 23.04.2025. Verfasst von Niki van Stein und 2 weiteren Autor:innen. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 31 Literaturangaben.
Abstract (Originalsprache)
Large Language Models (LLMs) have shown great potential in automatically generating and optimizing (meta)heuristics, making them valuable tools in heuristic optimization tasks. However, LLMs are generally inefficient when it comes to fine-tuning hyper-parameters of the generated algorithms, often requiring excessive queries that lead to high computational and financial costs. This paper presents a novel hybrid approach, LLaMEA-HPO, which integrates the open source LLaMEA (Large Language Model Evolutionary Algorithm) framework with a Hyper-Parameter Optimization (HPO) procedure in the loop. By offloading hyper-parameter tuning to an HPO procedure, the LLaMEA-HPO framework allows the LLM to focus on generating novel algorithmic structures, reducing the number of required LLM queries and improving the overall efficiency of the optimization process. We empirically validate the proposed hybrid framework on benchmark problems, including Online Bin Packing, Black-Box Optimization, and the Traveling Salesperson Problem. Our results demonstrate that LLaMEA-HPO achieves superior or comparable performance compared to existing LLM-driven frameworks while significantly reducing computational costs. This work highlights the importance of separating algorithmic innovation and structural code search from parameter tuning in LLM-driven code optimization and offers a scalable approach to improve the efficiency and effectiveness of LLM-based code generation.
Zitieren
APA
Stein, N. V., Vermetten, D., & Bäck, T. (2025). In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics. ACM Transactions on Evolutionary Learning and Optimization. https://doi.org/10.1145/3731567
Vancouver
Stein NV, Vermetten D, Bäck T. In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics. ACM Transactions on Evolutionary Learning and Optimization. 2025. doi:10.1145/3731567
BibTeX
@article{Stein2025,
author = {Stein, Niki van and Vermetten, Diederick and Bäck, Thomas},
title = {In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics},
journal = {ACM Transactions on Evolutionary Learning and Optimization},
year = {2025},
doi = {10.1145/3731567}
}
EUVIMED