Mathematical methods of reinforcement learning
Russian Mathematical Surveys · 2026 · 81 (4) · 583-669
DOI: 10.4213/rm10328e ↗Kurz gefasst
Fachartikel in der Zeitschrift Russian Mathematical Surveys, online erschienen am 10.11.2026. Verfasst von Denis Vital'evich Belomestny und 7 weiteren Autor:innen. Die Arbeit nennt 122 Literaturangaben. Gefördert von: National Research University Higher School of Economics, Ministry of Economic Development of the Russian Federation.
Abstract (Originalsprache)
Reinforcement learning (RL) is increasingly grounded in tools from probability, optimization, and operator theory. This survey organizes the mathematical structures that underpin the design and analysis of modern algorithms in RL. We begin from Markov decision processes (MDPs) and Bellman operators, emphasizing contraction mappings, monotonicity, and fixed-point theory that yield convergence guarantees and rates for value and policy iteration, and temporal-difference schemes. We then develop the optimization perspective: stochastic approximation and martingale methods, convex duality and the role of regularization linking mirror/proximal methods. Function approximation is treated through linear and non-linear settings, covering stabilization, error decomposition, and sample-complexity via concentration inequalities for dependent data and mixing processes. We further cover off-policy evaluation/learning, constrained RL, and constrained MDPs. Throughout, we unify algorithmic templates under common operator and variational lenses, highlighting both finite-sample bounds and asymptotic results. Our presentation is intended to provide a unified mathematical entry point for researchers in probability, optimization, and statistics who are interested in RL. Bibliography: 122 titles.
Förderung
- National Research University Higher School of Economics
- Ministry of Economic Development of the Russian Federation
Zitieren
APA
Belomestny, D. V., Gasnikov, A. V., Gladin, E. L., Naumov, A. A., Rubtsov, A. S., Sapronov, Y. F., Tyapkin, D. N., & Yudin, N. E. (2026). Mathematical methods of reinforcement learning. Russian Mathematical Surveys, 81(4), 583-669. https://doi.org/10.4213/rm10328e
Vancouver
Belomestny DV, Gasnikov AV, Gladin EL, Naumov AA, Rubtsov AS, Sapronov YF, et al. Mathematical methods of reinforcement learning. Russian Mathematical Surveys. 2026;81(4):583-669. doi:10.4213/rm10328e
BibTeX
@article{Belomestny2026,
author = {Belomestny, Denis Vital'evich and Gasnikov, Alexander Vladimirovich and Gladin, Egor Leonidovich and Naumov, Aleksei Aleksandrovich and Rubtsov, Artemy Sergeevich and Sapronov, Yurii Feliksovich and Tyapkin, Daniil Nikolaevich and Yudin, Nikita Evgen'evich},
title = {Mathematical methods of reinforcement learning},
journal = {Russian Mathematical Surveys},
year = {2026},
volume = {81},
number = {4},
pages = {583--669},
doi = {10.4213/rm10328e}
}
EUVIMED