EUVIMEDEUVIMED entdecken
Crossref

Convergence of Policy Iteration for Entropy-Regularized Stochastic Control Problems

Yu-Jui Huang, Zhenhua Wang, Zhou Zhou

SIAM Journal on Control and Optimization · 30.04.2025 · 63 (2) · 752-777

DOI: 10.1137/24m1638744 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Control and Optimization, online erschienen am 17.03.2025. Verfasst von Yu-Jui Huang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 29 Literaturangaben. Gefördert von: National Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

For a general entropy-regularized stochastic control problem on an infinite horizon, we prove that a policy iteration algorithm (PIA) converges to an optimal relaxed control. Contrary to the standard stochastic control literature, classical Hölder estimates of value functions do not ensure the convergence of the PIA, due to the added entropy-regularizing term. To circumvent this, we carry out a delicate estimation by moving back and forth between appropriate Hölder and Sobolev spaces. This requires new Sobolev estimates designed specifically for the purpose of policy iteration and a nontrivial technique to contain the entropy growth. Ultimately, we obtain a uniform Hölder bound for the sequence of value functions generated by the PIA, thereby achieving the desired convergence result. Characterization of the optimal value function as the unique solution to an exploratory Hamilton–Jacobi–Bellman equation comes as a by-product.

Förderung

  • National Science Foundation

Zitieren

APA

Huang, Y., Wang, Z., & Zhou, Z. (2025). Convergence of Policy Iteration for Entropy-Regularized Stochastic Control Problems. SIAM Journal on Control and Optimization, 63(2), 752-777. https://doi.org/10.1137/24m1638744

Vancouver

Huang Y, Wang Z, Zhou Z. Convergence of Policy Iteration for Entropy-Regularized Stochastic Control Problems. SIAM Journal on Control and Optimization. 2025;63(2):752-777. doi:10.1137/24m1638744

BibTeX
@article{Huang2025,
  author = {Huang, Yu-Jui and Wang, Zhenhua and Zhou, Zhou},
  title = {Convergence of Policy Iteration for Entropy-Regularized Stochastic Control Problems},
  journal = {SIAM Journal on Control and Optimization},
  year = {2025},
  volume = {63},
  number = {2},
  pages = {752--777},
  doi = {10.1137/24m1638744}
}
Kostenlos starten →