When to Explore and When to Exploit: Adaptive Decisions in Bayesian Optimization
Machine Learning and Knowledge Extraction · 2026-07-03 · 8 (7) · 193
DOI: 10.3390/make8070193 ↗At a glance
Journal article in Machine Learning and Knowledge Extraction, published online on 2026-07-03. Written by Antonio Candelieri and 2 co-authors. 75 references. The full text is freely available (open access, CC BY 4.0).
Abstract (original language)
Gaussian process-based Bayesian optimization (BO) is a sample-efficient sequential strategy for optimizing expensive black-box functions. The Gaussian process provides a probabilistic approximation of the unknown function, while an acquisition function balances exploration and exploitation to select the next evaluation point. Despite significant research efforts, no master acquisition function has been identified. This paper proposes a novel adaptive acquisition function that dynamically adjusts the exploration–exploitation trade-off based on the evolution of the optimization process, rather than using fixed or random scheduling. While implemented here within a GP-based BO framework, the core switching mechanism is surrogate-agnostic: the exploitative component requires only a surrogate point prediction, and the explorative component is entirely model-free. Unlike traditional approaches, where mechanisms like UCB/LCB lean toward exploration over iterations, or fixed strategies that switch from exploratory (EI) to exploitative (PI) behavior at predetermined points, the proposed method makes purely exploitative decisions using only the GP’s prediction. However, it discards these decisions when they have low potential for significant improvement, instead focusing on uncertainty reduction. Notably, this approach uses inverse distance weighting for uncertainty quantification rather than the GP’s predictive uncertainty, avoiding bias from the GP’s predictions. Testing on benchmark functions demonstrates that the proposed acquisition function is almost always Pareto optimal, offering the most balanced trade-off between convergence to the global optimum and exploration capability compared to state-of-the-art alternatives.
Cite
APA
Candelieri, A., Archetti, F., & Seyedi, I. (2026). When to Explore and When to Exploit: Adaptive Decisions in Bayesian Optimization. Machine Learning and Knowledge Extraction, 8(7), 193. https://doi.org/10.3390/make8070193
Vancouver
Candelieri A, Archetti F, Seyedi I. When to Explore and When to Exploit: Adaptive Decisions in Bayesian Optimization. Machine Learning and Knowledge Extraction. 2026;8(7):193. doi:10.3390/make8070193
BibTeX
@article{Candelieri2026,
author = {Candelieri, Antonio and Archetti, Francesco and Seyedi, Iman},
title = {When to Explore and When to Exploit: Adaptive Decisions in Bayesian Optimization},
journal = {Machine Learning and Knowledge Extraction},
year = {2026},
volume = {8},
number = {7},
pages = {193},
doi = {10.3390/make8070193}
}
EUVIMED