EUVIMEDOpdag EUVIMED
Crossref

Constrained Reinforcement Learning with Average Reward Objective: Model-Based and Model-Free Algorithms

Vaneet Aggarwal, Washim Uddin Mondal, Qinbo Bai

Foundations and Trends in Optimization · 2024

DOI: 10.1561/2400000038 ↗

At a glance

Journal article in Foundations and Trends in Optimization, 2024. Written by Vaneet Aggarwal and 2 co-authors. 72 references.

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Reinforcement Learning (RL) serves as a versatile framework for sequential decision-making, finding applications across diverse domains such as robotics, autonomous driving, recommendation systems, supply chain optimization, biology, mechanics, and finance. The primary objective of these applications is to maximize the average reward. Real-world scenarios often necessitate adherence to specific constraints during the learning process. This monograph focuses on the exploration of various model-based and model-free approaches for Constrained RL within the context of average reward Markov Decision Pro cesses (MDPs). The investigation commences with an examination of model-based strategies, delving into two foundational methods – optimism in the face of uncertainty and posterior sampling. Subsequently, the discussion transitions to parametrized model-free approaches, where the primal dual policy gradient-based algorithm is explored as a solution for constrained MDPs. The monograph provides regret guarantees and analyzes constraint violation for each of the discussed setups. For the above exploration, we assume the underlying MDP to be ergodic. Further, this monograph extends its discussion to encompass results tailored for weakly communicating MDPs, thereby broadening the scope of its findings and their relevance to a wider range of practical scenarios.

Cite

APA

Aggarwal, V., Mondal, W. U., & Bai, Q. (2024). Constrained Reinforcement Learning with Average Reward Objective: Model-Based and Model-Free Algorithms. Foundations and Trends in Optimization. https://doi.org/10.1561/2400000038

Vancouver

Aggarwal V, Mondal WU, Bai Q. Constrained Reinforcement Learning with Average Reward Objective: Model-Based and Model-Free Algorithms. Foundations and Trends in Optimization. 2024. doi:10.1561/2400000038

BibTeX
@article{Aggarwal2024,
  author = {Aggarwal, Vaneet and Mondal, Washim Uddin and Bai, Qinbo},
  title = {Constrained Reinforcement Learning with Average Reward Objective: Model-Based and Model-Free Algorithms},
  journal = {Foundations and Trends in Optimization},
  year = {2024},
  doi = {10.1561/2400000038}
}
Start for free →