EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges

Bernd Bischl, Martin Binder, Michel Lang, Tobias Pielok, Jakob Richter, Stefan Coors, Janek Thomas, Theresa Ullmann, Marc Becker, Anne‐Laure Boulesteix, Difan Deng, Marius Lindauer

WIREs Data Mining and Knowledge Discovery · 2023-03 · 13 (2)

DOI: 10.1002/widm.1484 ↗

Kurz gefasst

Fachartikel in der Zeitschrift WIREs Data Mining and Knowledge Discovery, online erschienen am 16.01.2023. Verfasst von Bernd Bischl und 11 weiteren Autor:innen. Die Arbeit wurde bisher 852-mal zitiert (Crossref) und nennt 174 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Bundesministerium für Bildung und Forschung, Deutsche Forschungsgemeinschaft.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Most machine learning algorithms are configured by a set of hyperparameters whose values must be carefully chosen and which often considerably impact performance. To avoid a time‐consuming and irreproducible manual process of trial‐and‐error to find well‐performing hyperparameter configurations, various automatic hyperparameter optimization (HPO) methods—for example, based on resampling error estimation for supervised machine learning—can be employed. After introducing HPO from a general perspective, this paper reviews important HPO methods, from simple techniques such as grid or random search to more advanced methods like evolution strategies, Bayesian optimization, Hyperband, and racing. This work gives practical recommendations regarding important choices to be made when conducting HPO, including the HPO algorithms themselves, performance evaluation, how to combine HPO with machine learning pipelines, runtime improvements, and parallelization. This article is categorized under: Algorithmic Development > Statistics Technologies > Machine Learning Technologies > Prediction

Förderung

  • Bundesministerium für Bildung und Forschung
  • Deutsche Forschungsgemeinschaft

Zitieren

APA

Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann, T., Becker, M., Boulesteix, A., Deng, D., & Lindauer, M. (2023). Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. WIREs Data Mining and Knowledge Discovery, 13(2). https://doi.org/10.1002/widm.1484

Vancouver

Bischl B, Binder M, Lang M, Pielok T, Richter J, Coors S, et al. Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. WIREs Data Mining and Knowledge Discovery. 2023;13(2). doi:10.1002/widm.1484

BibTeX
@article{Bischl2023,
  author = {Bischl, Bernd and Binder, Martin and Lang, Michel and Pielok, Tobias and Richter, Jakob and Coors, Stefan and Thomas, Janek and Ullmann, Theresa and Becker, Marc and Boulesteix, Anne‐Laure and Deng, Difan and Lindauer, Marius},
  title = {Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges},
  journal = {WIREs Data Mining and Knowledge Discovery},
  year = {2023},
  volume = {13},
  number = {2},
  doi = {10.1002/widm.1484}
}
Kostenlos starten →