EUVIMEDOdkrijte EUVIMED
Open access · CC BY 4.0

When to Explore and When to Exploit: Adaptive Decisions in Bayesian Optimization

Antonio Candelieri, Francesco Archetti, Iman Seyedi

Machine Learning and Knowledge Extraction · 2026-07-03 · 8 (7) · 193

DOI: 10.3390/make8070193 ↗

At a glance

Journal article in Machine Learning and Knowledge Extraction, published online on 2026-07-03. Written by Antonio Candelieri and 2 co-authors. 75 references. The full text is freely available (open access, CC BY 4.0).

PovzetekBibliografski podatkiCiteRelated studies

Abstract (original language)

Gaussian process-based Bayesian optimization (BO) is a sample-efficient sequential strategy for optimizing expensive black-box functions. The Gaussian process provides a probabilistic approximation of the unknown function, while an acquisition function balances exploration and exploitation to select the next evaluation point. Despite significant research efforts, no master acquisition function has been identified. This paper proposes a novel adaptive acquisition function that dynamically adjusts the exploration–exploitation trade-off based on the evolution of the optimization process, rather than using fixed or random scheduling. While implemented here within a GP-based BO framework, the core switching mechanism is surrogate-agnostic: the exploitative component requires only a surrogate point prediction, and the explorative component is entirely model-free. Unlike traditional approaches, where mechanisms like UCB/LCB lean toward exploration over iterations, or fixed strategies that switch from exploratory (EI) to exploitative (PI) behavior at predetermined points, the proposed method makes purely exploitative decisions using only the GP’s prediction. However, it discards these decisions when they have low potential for significant improvement, instead focusing on uncertainty reduction. Notably, this approach uses inverse distance weighting for uncertainty quantification rather than the GP’s predictive uncertainty, avoiding bias from the GP’s predictions. Testing on benchmark functions demonstrates that the proposed acquisition function is almost always Pareto optimal, offering the most balanced trade-off between convergence to the global optimum and exploration capability compared to state-of-the-art alternatives.

Cite

APA

Candelieri, A., Archetti, F., & Seyedi, I. (2026). When to Explore and When to Exploit: Adaptive Decisions in Bayesian Optimization. Machine Learning and Knowledge Extraction, 8(7), 193. https://doi.org/10.3390/make8070193

Vancouver

Candelieri A, Archetti F, Seyedi I. When to Explore and When to Exploit: Adaptive Decisions in Bayesian Optimization. Machine Learning and Knowledge Extraction. 2026;8(7):193. doi:10.3390/make8070193

BibTeX
@article{Candelieri2026,
  author = {Candelieri, Antonio and Archetti, Francesco and Seyedi, Iman},
  title = {When to Explore and When to Exploit: Adaptive Decisions in Bayesian Optimization},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {7},
  pages = {193},
  doi = {10.3390/make8070193}
}
Start for free →