Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges
WIREs Data Mining and Knowledge Discovery · 2023-03 · 13 (2)
DOI: 10.1002/widm.1484 ↗Kurz gefasst
Fachartikel in der Zeitschrift WIREs Data Mining and Knowledge Discovery, online erschienen am 16.01.2023. Verfasst von Bernd Bischl und 11 weiteren Autor:innen. Die Arbeit wurde bisher 852-mal zitiert (Crossref) und nennt 174 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Bundesministerium für Bildung und Forschung, Deutsche Forschungsgemeinschaft.
Abstract (Originalsprache)
Most machine learning algorithms are configured by a set of hyperparameters whose values must be carefully chosen and which often considerably impact performance. To avoid a time‐consuming and irreproducible manual process of trial‐and‐error to find well‐performing hyperparameter configurations, various automatic hyperparameter optimization (HPO) methods—for example, based on resampling error estimation for supervised machine learning—can be employed. After introducing HPO from a general perspective, this paper reviews important HPO methods, from simple techniques such as grid or random search to more advanced methods like evolution strategies, Bayesian optimization, Hyperband, and racing. This work gives practical recommendations regarding important choices to be made when conducting HPO, including the HPO algorithms themselves, performance evaluation, how to combine HPO with machine learning pipelines, runtime improvements, and parallelization. This article is categorized under: Algorithmic Development > Statistics Technologies > Machine Learning Technologies > Prediction
Förderung
- Bundesministerium für Bildung und Forschung
- Deutsche Forschungsgemeinschaft
Zitieren
APA
Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann, T., Becker, M., Boulesteix, A., Deng, D., & Lindauer, M. (2023). Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. WIREs Data Mining and Knowledge Discovery, 13(2). https://doi.org/10.1002/widm.1484
Vancouver
Bischl B, Binder M, Lang M, Pielok T, Richter J, Coors S, et al. Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. WIREs Data Mining and Knowledge Discovery. 2023;13(2). doi:10.1002/widm.1484
BibTeX
@article{Bischl2023,
author = {Bischl, Bernd and Binder, Martin and Lang, Michel and Pielok, Tobias and Richter, Jakob and Coors, Stefan and Thomas, Janek and Ullmann, Theresa and Becker, Marc and Boulesteix, Anne‐Laure and Deng, Difan and Lindauer, Marius},
title = {Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges},
journal = {WIREs Data Mining and Knowledge Discovery},
year = {2023},
volume = {13},
number = {2},
doi = {10.1002/widm.1484}
}
EUVIMED