EUVIMEDEUVIMED entdecken
Crossref

Accuracy of Discretely Sampled Stochastic Policies in Continuous-Time Reinforcement Learning

Yanwei Jia, Du Ouyang, Yufei Zhang

SIAM Journal on Control and Optimization · 30.06.2026 · 64 (3) · 1889-1929

DOI: 10.1137/25m1742199 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Control and Optimization, online erschienen am 11.06.2026. Verfasst von Yanwei Jia und 2 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 48 Literaturangaben. Gefördert von: Chinese University of Hong Kong, Research Grants Council, University Grants Committee, International Research Lab.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Stochastic policies (also known as relaxed controls) are widely used in continuous-time reinforcement learning algorithms. However, executing a stochastic policy and evaluating its performance in a continuous-time environment remain open challenges. This work introduces and rigorously analyzes a policy execution framework that samples actions from a stochastic policy at discrete time points and implements them as piecewise constant controls. We prove that as the sampling mesh size tends to zero, the controlled state process converges weakly to the dynamics with coefficients aggregated according to the stochastic policy. We explicitly quantify the convergence rate based on the regularity of the coefficients and establish an optimal first-order convergence rate for sufficiently regular coefficients. Additionally, we prove a [Formula: see text]-order weak convergence rate that holds uniformly over the sampling noise with high probability and establish a [Formula: see text]-order pathwise convergence for each realization of the system noise in the absence of volatility control. Building on these results, we analyze the bias and variance of various policy evaluation and policy gradient estimators based on discrete-time observations. Our results provide theoretical justification for the exploratory stochastic control framework in [H. Wang, T. Zariphopoulou, and X. Y. Zhou, J. Mach. Learn. Res., 21 (2020), pp. 1–34].

Förderung

  • Chinese University of Hong Kong
  • Research Grants Council, University Grants Committee
  • International Research Lab

Zitieren

APA

Jia, Y., Ouyang, D., & Zhang, Y. (2026). Accuracy of Discretely Sampled Stochastic Policies in Continuous-Time Reinforcement Learning. SIAM Journal on Control and Optimization, 64(3), 1889-1929. https://doi.org/10.1137/25m1742199

Vancouver

Jia Y, Ouyang D, Zhang Y. Accuracy of Discretely Sampled Stochastic Policies in Continuous-Time Reinforcement Learning. SIAM Journal on Control and Optimization. 2026;64(3):1889-1929. doi:10.1137/25m1742199

BibTeX
@article{Jia2026,
  author = {Jia, Yanwei and Ouyang, Du and Zhang, Yufei},
  title = {Accuracy of Discretely Sampled Stochastic Policies in Continuous-Time Reinforcement Learning},
  journal = {SIAM Journal on Control and Optimization},
  year = {2026},
  volume = {64},
  number = {3},
  pages = {1889--1929},
  doi = {10.1137/25m1742199}
}
Kostenlos starten →