EUVIMEDEUVIMED entdecken
Crossref

Sublinear Regret for a Class of Continuous-Time Linear-Quadratic Reinforcement Learning Problems

Yilie Huang, Yanwei Jia, Xun Yu Zhou

SIAM Journal on Control and Optimization · 31.10.2025 · 63 (5) · 3452-3474

DOI: 10.1137/24m1695075 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Control and Optimization, online erschienen am 17.09.2025. Verfasst von Yilie Huang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 63 Literaturangaben. Gefördert von: Chinese University of Hong Kong, Columbia University, Research Grants Council, University Grants Committee.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We study reinforcement learning (RL) for a class of continuous-time linear-quadratic (LQ) control problems for diffusions, where states are scalar-valued and running control rewards are absent but volatilities of the state processes depend on both state and control variables. We apply a model-free approach that relies neither on knowledge of model parameters nor on their estimations, and devise an RL algorithm to learn the optimal policy parameter directly. Our main contributions include the introduction of an exploration schedule and a regret analysis of the proposed algorithm. We provide the convergence rate of the policy parameter to the optimal one and prove that the algorithm achieves a regret bound of [Formula: see text] up to a logarithmic factor, where [Formula: see text] is the number of learning episodes. We conduct a simulation study to validate the theoretical results and demonstrate the effectiveness and reliability of the proposed algorithm. We also perform numerical comparisons between our method and those of the recent model-based stochastic LQ RL studies adapted to the state- and control-dependent volatility setting, demonstrating a better performance of the former in terms of regret bounds.

Förderung

  • Chinese University of Hong Kong
  • Columbia University
  • Research Grants Council, University Grants Committee

Zitieren

APA

Huang, Y., Jia, Y., & Zhou, X. Y. (2025). Sublinear Regret for a Class of Continuous-Time Linear-Quadratic Reinforcement Learning Problems. SIAM Journal on Control and Optimization, 63(5), 3452-3474. https://doi.org/10.1137/24m1695075

Vancouver

Huang Y, Jia Y, Zhou XY. Sublinear Regret for a Class of Continuous-Time Linear-Quadratic Reinforcement Learning Problems. SIAM Journal on Control and Optimization. 2025;63(5):3452-3474. doi:10.1137/24m1695075

BibTeX
@article{Huang2025,
  author = {Huang, Yilie and Jia, Yanwei and Zhou, Xun Yu},
  title = {Sublinear Regret for a Class of Continuous-Time Linear-Quadratic Reinforcement Learning Problems},
  journal = {SIAM Journal on Control and Optimization},
  year = {2025},
  volume = {63},
  number = {5},
  pages = {3452--3474},
  doi = {10.1137/24m1695075}
}
Kostenlos starten →