EUVIMEDEUVIMED entdecken
Crossref

Minimax Optimal Sample Complexity for Iterated CVaR Reinforcement Learning With a Generative Model

Zilong Deng, Alvaro Velasquez, Shaofeng Zou

IEEE Transactions on Information Theory · 2026-07 · 72 (7) · 5077-5103

DOI: 10.1109/tit.2026.3692816 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Information Theory, 2026-07. Verfasst von Zilong Deng und 2 weiteren Autor:innen. Gefördert von: NSF, Defense Advanced Research Projects Agency.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Standard Reinforcement Learning (RL) algorithms are typically designed to maximize the expected accumulative reward, which may be inadequate in scenarios where risk sensitivity is critical. In this work, the problem of risk-sensitive RL with Iterated Conditional Value at Risk is studied, where the objective is to optimize outcomes under a specified risk level $\tau $ at each step. This work provides the first minimax optimal sample complexity analysis for this problem with a generative model. Specifically, the sample complexity is firstly characterized as a function of the number of states $S$ , actions $A$ , and effective horizon $(1-\gamma)^{-1}$ (resp. horizon $H$ in the finite-horizon setting), and is further shown to be minimax optimal via a novel minimax lower bound analysis when the risk level $0\lt \tau \leq 1$ is treated as a constant or the risk level is above a threshold of $\gamma $ (resp. $1-1/H$ in the finite horizon setting). For the case when the risk level is small, the limiting case of $\tau \rightarrow 0$ , termed worst-path RL, is then studied, and the minimax optimal sample complexity is also theoretically characterized.

Förderung

  • NSF
  • Defense Advanced Research Projects Agency

Zitieren

APA

Deng, Z., Velasquez, A., & Zou, S. (2026). Minimax Optimal Sample Complexity for Iterated CVaR Reinforcement Learning With a Generative Model. IEEE Transactions on Information Theory, 72(7), 5077-5103. https://doi.org/10.1109/tit.2026.3692816

Vancouver

Deng Z, Velasquez A, Zou S. Minimax Optimal Sample Complexity for Iterated CVaR Reinforcement Learning With a Generative Model. IEEE Transactions on Information Theory. 2026;72(7):5077-5103. doi:10.1109/tit.2026.3692816

BibTeX
@article{Deng2026,
  author = {Deng, Zilong and Velasquez, Alvaro and Zou, Shaofeng},
  title = {Minimax Optimal Sample Complexity for Iterated CVaR Reinforcement Learning With a Generative Model},
  journal = {IEEE Transactions on Information Theory},
  year = {2026},
  volume = {72},
  number = {7},
  pages = {5077--5103},
  doi = {10.1109/tit.2026.3692816}
}
Kostenlos starten →