EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Entropy regularized reinforcement learning using large deviation theory

Argenis Arriojas, Jacob Adamczyk, Stas Tiomkin, Rahul V. Kulkarni

Physical Review Research · 10.05.2023 · 5 (2)

DOI: 10.1103/physrevresearch.5.023085 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Physical Review Research, online erschienen am 10.05.2023. Verfasst von Argenis Arriojas und 3 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Reinforcement learning (RL) is an important field of research in machine learning that is increasingly being applied to complex optimization problems in physics. In parallel, concepts from physics have contributed to important advances in RL with developments such as entropy-regularized RL. While these developments have led to advances in both fields, obtaining analytical solutions for optimization in entropy-regularized RL is currently an open problem. In this paper, we establish a mapping between entropy-regularized RL and research in non-equilibrium statistical mechanics focusing on Markovian processes conditioned on rare events. In the long-time limit, we apply approaches from large deviation theory to derive exact analytical results for the optimal policy and optimal dynamics in Markov Decision Process (MDP) models of reinforcement learning. The results obtained lead to a novel analytical and computational framework for entropy-regularized RL which is validated by simulations. The mapping established in this work connects current research in reinforcement learning and non-equilibrium statistical mechanics, thereby opening new avenues for the application of analytical and computational approaches from one field to cutting-edge problems in the other.

Förderung

  • National Science Foundation

Zitieren

APA

Arriojas, A., Adamczyk, J., Tiomkin, S., & Kulkarni, R. V. (2023). Entropy regularized reinforcement learning using large deviation theory. Physical Review Research, 5(2). https://doi.org/10.1103/physrevresearch.5.023085

Vancouver

Arriojas A, Adamczyk J, Tiomkin S, Kulkarni RV. Entropy regularized reinforcement learning using large deviation theory. Physical Review Research. 2023;5(2). doi:10.1103/physrevresearch.5.023085

BibTeX
@article{Arriojas2023,
  author = {Arriojas, Argenis and Adamczyk, Jacob and Tiomkin, Stas and Kulkarni, Rahul V.},
  title = {Entropy regularized reinforcement learning using large deviation theory},
  journal = {Physical Review Research},
  year = {2023},
  volume = {5},
  number = {2},
  doi = {10.1103/physrevresearch.5.023085}
}
Kostenlos starten →