Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control
SIAM Review · 31.12.2026 · 68 (4) · 823-849
DOI: 10.1137/25m1782856 ↗Kurz gefasst
Fachartikel in der Zeitschrift SIAM Review, online erschienen am 12.11.2026. Verfasst von René Carmona und 1 weiteren Autor:innen. Die Arbeit nennt 51 Literaturangaben. Gefördert von: Air Force Office of Scientific Research.
Abstract (Originalsprache)
Reinforcement learning (RL) is currently one of the most prominent methods for optimizing dynamical systems, with breakthrough results across various fields. The framework is based on the concept of a Markov decision process (MDP), leading to a discrete-time optimal control problem. In the RL literature, such problems are typically formulated and solved using mixed policies, from which random actions are sampled at each time step. Recently, part of the optimal control community has begun investigating continuous-time versions of RL algorithms, replacing MDPs with continuous-time stochastic processes governed by relaxed controls and asserting a full analogy between the two formulations. In this work, we examine the limitations of this analogy and rigorously establish a connection between the two problems in the case where only the drift term of the continuous-time model is controlled. We prove strong convergence of the RL implementation of mixed strategies as the time discretization mesh tends to zero. We also discuss the technical challenges posed by the possible presence of control in the diffusion component of the state.
Förderung
- Air Force Office of Scientific Research
Zitieren
APA
Carmona, R., & Laurière, M. (2026). Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control. SIAM Review, 68(4), 823-849. https://doi.org/10.1137/25m1782856
Vancouver
Carmona R, Laurière M. Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control. SIAM Review. 2026;68(4):823-849. doi:10.1137/25m1782856
BibTeX
@article{Carmona2026,
author = {Carmona, René and Laurière, Mathieu},
title = {Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control},
journal = {SIAM Review},
year = {2026},
volume = {68},
number = {4},
pages = {823--849},
doi = {10.1137/25m1782856}
}
EUVIMED