EUVIMEDEUVIMED entdecken
Crossref

Stochastic Optimization Methods for Policy Evaluation in Reinforcement Learning

Yi Zhou, Shacong Ma

Foundations and Trends in Optimization · 2024

DOI: 10.1561/2400000045 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Foundations and Trends in Optimization, 2024. Verfasst von Yi Zhou und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 49 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This monograph introduces various value-based approaches for solving the policy evaluation problem in the online reinforcement learning (RL) scenario, which aims to learn the value function associated with a specific policy under a single Markov decision process (MDP). Approaches vary depending on whether they are implemented in an on-policy or off-policy manner: In on-policy settings, where the evaluation of the policy is conducted using data generated from the same policy that is being assessed, classical techniques such as TD(0), TD(λ), and their extensions with function approximation or variance reduction are employed in this setting. For off-policy evaluation, where samples are collected under a different behavior policy, this monograph introduces gradient-based two-timescale algorithms like GTD2, TDC, and variance-reduced TDC. These algorithms are designed to minimize the mean-squared projected Bellman error (MSPBE) as the objective function. This monograph also discusses their finite-sample convergence upper bounds and sample complexity.

Zitieren

APA

Zhou, Y., & Ma, S. (2024). Stochastic Optimization Methods for Policy Evaluation in Reinforcement Learning. Foundations and Trends in Optimization. https://doi.org/10.1561/2400000045

Vancouver

Zhou Y, Ma S. Stochastic Optimization Methods for Policy Evaluation in Reinforcement Learning. Foundations and Trends in Optimization. 2024. doi:10.1561/2400000045

BibTeX
@article{Zhou2024,
  author = {Zhou, Yi and Ma, Shacong},
  title = {Stochastic Optimization Methods for Policy Evaluation in Reinforcement Learning},
  journal = {Foundations and Trends in Optimization},
  year = {2024},
  doi = {10.1561/2400000045}
}
Kostenlos starten →