EUVIMEDEUVIMED entdecken
Crossref

Policy Newton Methods for Distortion Riskmetrics

Soumen Pachal, Mizhaan Prajit Maniyar, Prashanth L. A.

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (29) · 24674-24681

DOI: 10.1609/aaai.v40i29.39652 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Soumen Pachal und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We consider the problem of risk-sensitive control in a reinforcement learning (RL) framework. In particular, we aim to find a risk-optimal policy by maximizing the distortion riskmetric (DRM) of the discounted reward in a finite-horizon Markov decision process (MDP). DRMs are a rich class of risk measures that include several well-known risk measures as special cases. We derive a policy Hessian theorem for the DRM objective using the likelihood ratio method. Using this result, we propose a natural DRM Hessian estimator from sample trajectories of the underlying MDP. Next, we present a cubic-regularized policy Newton algorithm for solving this problem in an on-policy RL setting using estimates of the DRM gradient and Hessian. Our proposed algorithm is shown to converge to an ϵ-second-order stationary point (ϵ-SOSP) of the DRM objective, and this guarantee ensures the escaping of saddle points. The sample complexity of our algorithms to find an ϵ-SOSP is O(ϵ−3.5). Our experiments validate the theoretical findings. To the best of our knowledge, our is the first work to present convergence to an ϵ-SOSP of a risk-sensitive objective, while existing works in the literature have either shown convergence to a first-order stationary point of a risk-sensitive objective, or a SOSP of a risk-neutral one.

Zitieren

APA

Pachal, S., Maniyar, M. P., & A., P. L. (2026). Policy Newton Methods for Distortion Riskmetrics. Proceedings of the AAAI Conference on Artificial Intelligence, 40(29), 24674-24681. https://doi.org/10.1609/aaai.v40i29.39652

Vancouver

Pachal S, Maniyar MP, A. PL. Policy Newton Methods for Distortion Riskmetrics. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(29):24674-24681. doi:10.1609/aaai.v40i29.39652

BibTeX
@article{Pachal2026,
  author = {Pachal, Soumen and Maniyar, Mizhaan Prajit and A., Prashanth L.},
  title = {Policy Newton Methods for Distortion Riskmetrics},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {29},
  pages = {24674--24681},
  doi = {10.1609/aaai.v40i29.39652}
}
Kostenlos starten →