EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Ergodicity in reinforcement learning

Dominik Baumann, Erfaun Noorani, Arsenii Mustafin, Xinyi Sheng, Bert Verbruggen, Arne Vanhoyweghen, Vincent Ginis, Thomas B. Schön

Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · 19.11.2026 · 384 (2334)

DOI: 10.1098/rsta.2025.0156 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, online erschienen am 19.11.2026. Verfasst von Dominik Baumann und 7 weiteren Autor:innen. Die Arbeit nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Finnish Ministry of Education and Culture, Research Council of Finland Flagship programme: Finnish Center for Artificial Intelligence FCAI, Kjell och Märta Beijer Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In reinforcement learning, we typically aim to optimize the expected value of the sum of rewards an agent collects over a trajectory. However, if the process generating these rewards is non-ergodic, the expected value, i.e. the average over infinitely many trajectories with a given policy, is uninformative for the average over a single, but infinitely long trajectory. Thus, if we care about how the individual agent performs during deployment, the expected value is not a good optimization objective. In this paper, we discuss the impact of non-ergodic reward processes on reinforcement learning agents through an instructive example, relate the notion of ergodic reward processes to more widely used notions of ergodic Markov chains, and present existing solutions that optimize long-term performance of individual trajectories under non-ergodic reward dynamics. This article is part of the theme issue ‘Ergodicity and ergodicity breaking in the sciences’.

Förderung

  • Finnish Ministry of Education and Culture
  • Research Council of Finland Flagship programme: Finnish Center for Artificial Intelligence FCAI
  • Kjell och Märta Beijer Foundation

Zitieren

APA

Baumann, D., Noorani, E., Mustafin, A., Sheng, X., Verbruggen, B., Vanhoyweghen, A., Ginis, V., & Schön, T. B. (2026). Ergodicity in reinforcement learning. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 384(2334). https://doi.org/10.1098/rsta.2025.0156

Vancouver

Baumann D, Noorani E, Mustafin A, Sheng X, Verbruggen B, Vanhoyweghen A, et al. Ergodicity in reinforcement learning. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2026;384(2334). doi:10.1098/rsta.2025.0156

BibTeX
@article{Baumann2026,
  author = {Baumann, Dominik and Noorani, Erfaun and Mustafin, Arsenii and Sheng, Xinyi and Verbruggen, Bert and Vanhoyweghen, Arne and Ginis, Vincent and Schön, Thomas B.},
  title = {Ergodicity in reinforcement learning},
  journal = {Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences},
  year = {2026},
  volume = {384},
  number = {2334},
  doi = {10.1098/rsta.2025.0156}
}
Kostenlos starten →