EUVIMEDEUVIMED entdecken
Crossref

Choquet Regularization for Continuous-Time Reinforcement Learning

Xia Han, Ruodu Wang, Xun Yu Zhou

SIAM Journal on Control and Optimization · 31.10.2023 · 61 (5) · 2777-2801

DOI: 10.1137/22m1524734 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Control and Optimization, online erschienen am 14.09.2023. Verfasst von Xia Han und 2 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: Columbia University, Natural Sciences and Engineering Research Council of Canada, Nankai University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We propose Choquet regularizers to measure and manage the level of exploration for reinforcement learning (RL) and reformulate the continuous-time entropy-regularized RL problem of H. Wang, T. Zariphopoulou, and X. Zhou [ J. Mach. Learn. Res., 21 (2020), pp. 1–34] in which we replace the differential entropy used for regularization with a Choquet regularizer. We derive the Hamilton–Jacobi–Bellman equation of the problem and solve it explicitly in the linear-quadratic (LQ) case via maximizing statically a mean-variance constrained Choquet regularizer. Under the LQ setting, we derive explicit optimal distributions for several specific Choquet regularizers and conversely identify the Choquet regularizers that generate a number of broadly used exploratory samplers, such as [Formula: see text]-greedy, exponential, uniform, and Gaussian.

Förderung

  • Columbia University
  • Natural Sciences and Engineering Research Council of Canada
  • Nankai University

Zitieren

APA

Han, X., Wang, R., & Zhou, X. Y. (2023). Choquet Regularization for Continuous-Time Reinforcement Learning. SIAM Journal on Control and Optimization, 61(5), 2777-2801. https://doi.org/10.1137/22m1524734

Vancouver

Han X, Wang R, Zhou XY. Choquet Regularization for Continuous-Time Reinforcement Learning. SIAM Journal on Control and Optimization. 2023;61(5):2777-2801. doi:10.1137/22m1524734

BibTeX
@article{Han2023,
  author = {Han, Xia and Wang, Ruodu and Zhou, Xun Yu},
  title = {Choquet Regularization for Continuous-Time Reinforcement Learning},
  journal = {SIAM Journal on Control and Optimization},
  year = {2023},
  volume = {61},
  number = {5},
  pages = {2777--2801},
  doi = {10.1137/22m1524734}
}
Kostenlos starten →