Choquet Regularization for Continuous-Time Reinforcement Learning
SIAM Journal on Control and Optimization · 31.10.2023 · 61 (5) · 2777-2801
DOI: 10.1137/22m1524734 ↗Kurz gefasst
Fachartikel in der Zeitschrift SIAM Journal on Control and Optimization, online erschienen am 14.09.2023. Verfasst von Xia Han und 2 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: Columbia University, Natural Sciences and Engineering Research Council of Canada, Nankai University.
Abstract (Originalsprache)
We propose Choquet regularizers to measure and manage the level of exploration for reinforcement learning (RL) and reformulate the continuous-time entropy-regularized RL problem of H. Wang, T. Zariphopoulou, and X. Zhou [ J. Mach. Learn. Res., 21 (2020), pp. 1–34] in which we replace the differential entropy used for regularization with a Choquet regularizer. We derive the Hamilton–Jacobi–Bellman equation of the problem and solve it explicitly in the linear-quadratic (LQ) case via maximizing statically a mean-variance constrained Choquet regularizer. Under the LQ setting, we derive explicit optimal distributions for several specific Choquet regularizers and conversely identify the Choquet regularizers that generate a number of broadly used exploratory samplers, such as [Formula: see text]-greedy, exponential, uniform, and Gaussian.
Förderung
- Columbia University
- Natural Sciences and Engineering Research Council of Canada
- Nankai University
Zitieren
APA
Han, X., Wang, R., & Zhou, X. Y. (2023). Choquet Regularization for Continuous-Time Reinforcement Learning. SIAM Journal on Control and Optimization, 61(5), 2777-2801. https://doi.org/10.1137/22m1524734
Vancouver
Han X, Wang R, Zhou XY. Choquet Regularization for Continuous-Time Reinforcement Learning. SIAM Journal on Control and Optimization. 2023;61(5):2777-2801. doi:10.1137/22m1524734
BibTeX
@article{Han2023,
author = {Han, Xia and Wang, Ruodu and Zhou, Xun Yu},
title = {Choquet Regularization for Continuous-Time Reinforcement Learning},
journal = {SIAM Journal on Control and Optimization},
year = {2023},
volume = {61},
number = {5},
pages = {2777--2801},
doi = {10.1137/22m1524734}
}
EUVIMED