EUVIMEDEUVIMED entdecken
Crossref

Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning

Avik Kar, Rahul Singh

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (27) · 22527-22535

DOI: 10.1609/aaai.v40i27.39412 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Avik Kar und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We study infinite-horizon average-reward reinforcement learning for continuous space Lipschitz Markov decision processes (MDPs) in which an agent can play policies from a given set Φ. The proposed algorithms efficiently explore the policy space by “zooming” into the “promising regions” of Φ, thereby achieving adaptivity gains in the performance. We upper bound the regret as O ̃(T^(1-d_(eff.)^(-1) ) ), where d_(eff.) = d_z^Φ+2 for our model-free algorithm PZRL-MF and d_(eff.) = 2d_S + d_z^Φ+ 3 for our model-based algorithm PZRL-MB. Here, d_S is the dimension of the state space, and d_z^Φ is the zooming dimension given a set of policies Φ. d_z^Φ is an alternative measure of the complexity of the problem, and it depends on the underlying MDP as well as on Φ. Hence, the proposed algorithms exhibit low regret in case the problem instance is benign and/or the agent competes against a low-complexity Φ (that has a small d_z^Φ). When specialized to the case of finite-dimensional policy space, we obtain that d_(eff.) scales as the dimension of this space under mild technical conditions; and also obtain d_(eff.) = 2, or equivalently O ̃(√T) regret for PZRL-MF, under a curvature condition on the average reward function that is commonly used in the multi-armed bandit (MAB) literature.

Zitieren

APA

Kar, A., & Singh, R. (2026). Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(27), 22527-22535. https://doi.org/10.1609/aaai.v40i27.39412

Vancouver

Kar A, Singh R. Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(27):22527-22535. doi:10.1609/aaai.v40i27.39412

BibTeX
@article{Kar2026,
  author = {Kar, Avik and Singh, Rahul},
  title = {Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {27},
  pages = {22527--22535},
  doi = {10.1609/aaai.v40i27.39412}
}
Kostenlos starten →