Multi-Objective Hyperparameter Optimization in Machine Learning—An Overview
ACM Transactions on Evolutionary Learning and Optimization · 31.12.2023 · 3 (4) · 1-50
DOI: 10.1145/3610536 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Evolutionary Learning and Optimization, online erschienen am 12.12.2023. Verfasst von Florian Karl und 12 weiteren Autor:innen. Die Arbeit wurde bisher 119-mal zitiert (Crossref) und nennt 271 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Bavarian Ministry of Economic Affairs, Regional Development and Energy through the Center for Analytics–Data–Applications (ADA-Center) within the framework of…, German Federal Ministry of Education and Research.
Abstract (Originalsprache)
Hyperparameter optimization constitutes a large part of typical modern machine learning (ML) workflows. This arises from the fact that ML methods and corresponding preprocessing steps often only yield optimal performance when hyperparameters are properly tuned. But in many applications, we are not only interested in optimizing ML pipelines solely for predictive accuracy; additional metrics or constraints must be considered when determining an optimal configuration, resulting in a multi-objective optimization problem. This is often neglected in practice, due to a lack of knowledge and readily available software implementations for multi-objective hyperparameter optimization. In this work, we introduce the reader to the basics of multi-objective hyperparameter optimization and motivate its usefulness in applied ML. Furthermore, we provide an extensive survey of existing optimization strategies from the domains of evolutionary algorithms and Bayesian optimization. We illustrate the utility of multi-objective optimization in several specific ML applications, considering objectives such as operating conditions, prediction time, sparseness, fairness, interpretability, and robustness.
Förderung
- Bavarian Ministry of Economic Affairs, Regional Development and Energy through the Center for Analytics–Data–Applications (ADA-Center) within the framework of…
- German Federal Ministry of Education and Research
Zitieren
APA
Karl, F., Pielok, T., Moosbauer, J., Pfisterer, F., Coors, S., Binder, M., Schneider, L., Thomas, J., Richter, J., Lang, M., Garrido-Merchán, E. C., Branke, J., & Bischl, B. (2023). Multi-Objective Hyperparameter Optimization in Machine Learning—An Overview. ACM Transactions on Evolutionary Learning and Optimization, 3(4), 1-50. https://doi.org/10.1145/3610536
Vancouver
Karl F, Pielok T, Moosbauer J, Pfisterer F, Coors S, Binder M, et al. Multi-Objective Hyperparameter Optimization in Machine Learning—An Overview. ACM Transactions on Evolutionary Learning and Optimization. 2023;3(4):1-50. doi:10.1145/3610536
BibTeX
@article{Karl2023,
author = {Karl, Florian and Pielok, Tobias and Moosbauer, Julia and Pfisterer, Florian and Coors, Stefan and Binder, Martin and Schneider, Lennart and Thomas, Janek and Richter, Jakob and Lang, Michel and Garrido-Merchán, Eduardo C. and Branke, Juergen and Bischl, Bernd},
title = {Multi-Objective Hyperparameter Optimization in Machine Learning—An Overview},
journal = {ACM Transactions on Evolutionary Learning and Optimization},
year = {2023},
volume = {3},
number = {4},
pages = {1--50},
doi = {10.1145/3610536}
}
EUVIMED