EUVIMEDEUVIMED entdecken
Crossref

Asymptotically optimal policies for weakly coupled Markov decision processes

Diego Goldsztajn, Konstantin Avrachenkov

Journal of Applied Probability · 2026-09 · 63 (3) · 1213-1243

DOI: 10.1017/jpr.2026.10079 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Applied Probability, online erschienen am 25.03.2026. Verfasst von Diego Goldsztajn und 1 weiteren Autor:innen. Die Arbeit nennt 29 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We consider the problem of maximizing the expected average reward obtained over an infinite time horizon by n weakly coupled Markov decision processes. Our setup is a substantial generalization of the multi-armed restless bandit problem that allows for multiple actions and constraints. We establish a connection with a deterministic and continuous-variable control problem where the objective is to maximize the average reward derived from an occupancy measure that represents the empirical distribution of the processes when n right arrow normal infinity n → ∞ $n \to \infty$ . We show that a solution of this fluid problem can be used to construct policies for the weakly coupled processes that achieve the maximum expected average reward as n right arrow normal infinity n → ∞ $n \to \infty$ , and we give sufficient conditions for the existence of solutions. Under certain assumptions on the constraints, we prove that these conditions are automatically satisfied if the unconstrained single-process problem admits a suitable unichain and aperiodic policy. In particular, the assumptions include multi-armed restless bandits and a broad class of problems with multiple actions and inequality constraints. Also, the policies can be constructed in an explicit way in these cases. Our theoretical results are complemented by several concrete examples and numerical experiments, which include multichain setups that are covered by the theoretical results.

Zitieren

APA

Goldsztajn, D., & Avrachenkov, K. (2026). Asymptotically optimal policies for weakly coupled Markov decision processes. Journal of Applied Probability, 63(3), 1213-1243. https://doi.org/10.1017/jpr.2026.10079

Vancouver

Goldsztajn D, Avrachenkov K. Asymptotically optimal policies for weakly coupled Markov decision processes. Journal of Applied Probability. 2026;63(3):1213-1243. doi:10.1017/jpr.2026.10079

BibTeX
@article{Goldsztajn2026,
  author = {Goldsztajn, Diego and Avrachenkov, Konstantin},
  title = {Asymptotically optimal policies for weakly coupled Markov decision processes},
  journal = {Journal of Applied Probability},
  year = {2026},
  volume = {63},
  number = {3},
  pages = {1213--1243},
  doi = {10.1017/jpr.2026.10079}
}
Kostenlos starten →