Risk‐sensitive Markov decision processes with long‐run CVaR criterion
Production and Operations Management · 2023-12 · 32 (12) · 4049-4067
DOI: 10.1111/poms.14077 ↗Kurz gefasst
Fachartikel in der Zeitschrift Production and Operations Management, 2023-12. Verfasst von Li Xia und 2 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 42 Literaturangaben. Gefördert von: National Key Research and Development Program of China, Guangdong Basic and Applied Basic Research Foundation, Guangdong Province Key Laboratory of Computational Science.
Abstract (Originalsprache)
CVaR (Conditional value at risk) is a risk metric widely used in finance. However, dynamically optimizing CVaR is difficult, because it is not a standard Markov decision process (MDP) and the principle of dynamic programming fails. In this paper, we study the infinite‐horizon discrete‐time MDP with a long‐run CVaR criterion, from the view of sensitivity‐based optimization. By introducing a pseudo‐CVaR metric, we reformulate the problem as a bilevel MDP model and derive a CVaR difference formula that quantifies the difference of long‐run CVaR under any two policies. The optimality of deterministic policies is derived. We obtain a so‐called Bellman local optimality equation for CVaR, which is a necessary and sufficient condition for locally optimal policies and only necessary for globally optimal policies. A CVaR derivative formula is also derived for providing more sensitivity information. Then we develop a policy iteration type algorithm to efficiently optimize CVaR, which is shown to converge to a local optimum in mixed policy space. Furthermore, based on the sensitivity analysis of our bilevel MDP formulation and critical points, we develop a globally optimal algorithm. The piecewise linearity and segment convexity of the optimal pseudo‐CVaR function are also established. Our main results and algorithms are further extended to optimize the mean and CVaR simultaneously. Finally, we conduct numerical experiments relating to portfolio management to demonstrate the main results. Our work sheds light on dynamically optimizing CVaR from a sensitivity viewpoint.
Förderung
- National Key Research and Development Program of China
- Guangdong Basic and Applied Basic Research Foundation
- Guangdong Province Key Laboratory of Computational Science
- Fundamental Research Funds for the Central Universities
- InnoHK
- The Government of the HKSAR
- Laboratory for AI‐Powered Financial Technologies
- National Natural Science Foundation of China
Zitieren
APA
Xia, L., Zhang, L., & Glynn, P. W. (2023). Risk‐sensitive Markov decision processes with long‐run CVaR criterion. Production and Operations Management, 32(12), 4049-4067. https://doi.org/10.1111/poms.14077
Vancouver
Xia L, Zhang L, Glynn PW. Risk‐sensitive Markov decision processes with long‐run CVaR criterion. Production and Operations Management. 2023;32(12):4049-4067. doi:10.1111/poms.14077
BibTeX
@article{Xia2023,
author = {Xia, Li and Zhang, Luyao and Glynn, Peter W.},
title = {Risk‐sensitive Markov decision processes with long‐run CVaR criterion},
journal = {Production and Operations Management},
year = {2023},
volume = {32},
number = {12},
pages = {4049--4067},
doi = {10.1111/poms.14077}
}
EUVIMED