EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Agent-Based Collaborative Random Search for Hyperparameter Tuning and Global Function Optimization

Ahmad Esmaeili, Zahra Ghorrati, Eric T. Matson

Systems · 05.05.2023 · 11 (5) · 228

DOI: 10.3390/systems11050228 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Systems, online erschienen am 05.05.2023. Verfasst von Ahmad Esmaeili und 2 weiteren Autor:innen. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Hyperparameter optimization is one of the most tedious yet crucial steps in training machine learning models. There are numerous methods for this vital model-building stage, ranging from domain-specific manual tuning guidelines suggested by the oracles to the utilization of general purpose black-box optimization techniques. This paper proposes an agent-based collaborative technique for finding near-optimal values for any arbitrary set of hyperparameters (or decision variables) in a machine learning model (or a black-box function optimization problem). The developed method forms a hierarchical agent-based architecture for the distribution of the searching operations at different dimensions and employs a cooperative searching procedure based on an adaptive width-based random sampling technique to locate the optima. The behavior of the presented model, specifically against changes in its design parameters, is investigated in both machine learning and global function optimization applications, and its performance is compared with that of two randomized tuning strategies that are commonly used in practice. Moreover, we have compared the performance of the proposed approach against particle swarm optimization (PSO) and simulated annealing (SA) methods in function optimization to provide additional insights into its exploration in the search space. According to the empirical results, the proposed model outperformed the compared random-based methods in almost all tasks conducted, notably in a higher number of dimensions and in the presence of limited on-device computational resources.

Zitieren

APA

Esmaeili, A., Ghorrati, Z., & Matson, E. T. (2023). Agent-Based Collaborative Random Search for Hyperparameter Tuning and Global Function Optimization. Systems, 11(5), 228. https://doi.org/10.3390/systems11050228

Vancouver

Esmaeili A, Ghorrati Z, Matson ET. Agent-Based Collaborative Random Search for Hyperparameter Tuning and Global Function Optimization. Systems. 2023;11(5):228. doi:10.3390/systems11050228

BibTeX
@article{Esmaeili2023,
  author = {Esmaeili, Ahmad and Ghorrati, Zahra and Matson, Eric T.},
  title = {Agent-Based Collaborative Random Search for Hyperparameter Tuning and Global Function Optimization},
  journal = {Systems},
  year = {2023},
  volume = {11},
  number = {5},
  pages = {228},
  doi = {10.3390/systems11050228}
}
Kostenlos starten →