EUVIMEDEUVIMED entdecken
Crossref

Logarithmic Regret Bounds for Continuous-Time Average-Reward Markov Decision Processes

Xuefeng Gao, Xun Yu Zhou

SIAM Journal on Control and Optimization · 31.10.2024 · 62 (5) · 2529-2556

DOI: 10.1137/23m1584101 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Control and Optimization, online erschienen am 10.09.2024. Verfasst von Xuefeng Gao und 1 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 38 Literaturangaben. Gefördert von: Hong Kong Research Grant Council, Nie Center for Intelligent Asset Management.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We consider reinforcement learning for continuous-time Markov decision processes (MDPs) in the infinite-horizon, average-reward setting. In contrast to discrete-time MDPs, a continuous-time process moves to a state and stays there for a random holding time after an action is taken. With unknown transition probabilities and rates of exponential holding times, we derive instance-dependent regret lower bounds that are logarithmic in the time horizon. Moreover, we design a learning algorithm and establish a finite-time regret bound that achieves the logarithmic growth rate. Our analysis builds upon upper confidence reinforcement learning, a delicate estimation of the mean holding times, and stochastic comparison of point processes.

Förderung

  • Hong Kong Research Grant Council
  • Nie Center for Intelligent Asset Management

Zitieren

APA

Gao, X., & Zhou, X. Y. (2024). Logarithmic Regret Bounds for Continuous-Time Average-Reward Markov Decision Processes. SIAM Journal on Control and Optimization, 62(5), 2529-2556. https://doi.org/10.1137/23m1584101

Vancouver

Gao X, Zhou XY. Logarithmic Regret Bounds for Continuous-Time Average-Reward Markov Decision Processes. SIAM Journal on Control and Optimization. 2024;62(5):2529-2556. doi:10.1137/23m1584101

BibTeX
@article{Gao2024,
  author = {Gao, Xuefeng and Zhou, Xun Yu},
  title = {Logarithmic Regret Bounds for Continuous-Time Average-Reward Markov Decision Processes},
  journal = {SIAM Journal on Control and Optimization},
  year = {2024},
  volume = {62},
  number = {5},
  pages = {2529--2556},
  doi = {10.1137/23m1584101}
}
Kostenlos starten →