EUVIMEDEUVIMED entdecken
Crossref

Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control

René Carmona, Mathieu Laurière

SIAM Review · 31.12.2026 · 68 (4) · 823-849

DOI: 10.1137/25m1782856 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Review, online erschienen am 12.11.2026. Verfasst von René Carmona und 1 weiteren Autor:innen. Die Arbeit nennt 51 Literaturangaben. Gefördert von: Air Force Office of Scientific Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Reinforcement learning (RL) is currently one of the most prominent methods for optimizing dynamical systems, with breakthrough results across various fields. The framework is based on the concept of a Markov decision process (MDP), leading to a discrete-time optimal control problem. In the RL literature, such problems are typically formulated and solved using mixed policies, from which random actions are sampled at each time step. Recently, part of the optimal control community has begun investigating continuous-time versions of RL algorithms, replacing MDPs with continuous-time stochastic processes governed by relaxed controls and asserting a full analogy between the two formulations. In this work, we examine the limitations of this analogy and rigorously establish a connection between the two problems in the case where only the drift term of the continuous-time model is controlled. We prove strong convergence of the RL implementation of mixed strategies as the time discretization mesh tends to zero. We also discuss the technical challenges posed by the possible presence of control in the diffusion component of the state.

Förderung

  • Air Force Office of Scientific Research

Zitieren

APA

Carmona, R., & Laurière, M. (2026). Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control. SIAM Review, 68(4), 823-849. https://doi.org/10.1137/25m1782856

Vancouver

Carmona R, Laurière M. Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control. SIAM Review. 2026;68(4):823-849. doi:10.1137/25m1782856

BibTeX
@article{Carmona2026,
  author = {Carmona, René and Laurière, Mathieu},
  title = {Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control},
  journal = {SIAM Review},
  year = {2026},
  volume = {68},
  number = {4},
  pages = {823--849},
  doi = {10.1137/25m1782856}
}
Kostenlos starten →