Logarithmic Regret Bounds for Continuous-Time Average-Reward Markov Decision Processes
SIAM Journal on Control and Optimization · 31.10.2024 · 62 (5) · 2529-2556
DOI: 10.1137/23m1584101 ↗Kurz gefasst
Fachartikel in der Zeitschrift SIAM Journal on Control and Optimization, online erschienen am 10.09.2024. Verfasst von Xuefeng Gao und 1 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 38 Literaturangaben. Gefördert von: Hong Kong Research Grant Council, Nie Center for Intelligent Asset Management.
Abstract (Originalsprache)
We consider reinforcement learning for continuous-time Markov decision processes (MDPs) in the infinite-horizon, average-reward setting. In contrast to discrete-time MDPs, a continuous-time process moves to a state and stays there for a random holding time after an action is taken. With unknown transition probabilities and rates of exponential holding times, we derive instance-dependent regret lower bounds that are logarithmic in the time horizon. Moreover, we design a learning algorithm and establish a finite-time regret bound that achieves the logarithmic growth rate. Our analysis builds upon upper confidence reinforcement learning, a delicate estimation of the mean holding times, and stochastic comparison of point processes.
Förderung
- Hong Kong Research Grant Council
- Nie Center for Intelligent Asset Management
Zitieren
APA
Gao, X., & Zhou, X. Y. (2024). Logarithmic Regret Bounds for Continuous-Time Average-Reward Markov Decision Processes. SIAM Journal on Control and Optimization, 62(5), 2529-2556. https://doi.org/10.1137/23m1584101
Vancouver
Gao X, Zhou XY. Logarithmic Regret Bounds for Continuous-Time Average-Reward Markov Decision Processes. SIAM Journal on Control and Optimization. 2024;62(5):2529-2556. doi:10.1137/23m1584101
BibTeX
@article{Gao2024,
author = {Gao, Xuefeng and Zhou, Xun Yu},
title = {Logarithmic Regret Bounds for Continuous-Time Average-Reward Markov Decision Processes},
journal = {SIAM Journal on Control and Optimization},
year = {2024},
volume = {62},
number = {5},
pages = {2529--2556},
doi = {10.1137/23m1584101}
}
EUVIMED