Relaxed Policy Iteration Algorithm for Nonlinear Zero-Sum Games With Application to H-Infinity Control
IEEE Transactions on Automatic Control · 2024-01 · 69 (1) · 426-433
DOI: 10.1109/tac.2023.3266277 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Automatic Control, 2024-01. Verfasst von Jie Li und 6 weiteren Autor:innen. Die Arbeit wurde bisher 36-mal zitiert (Crossref) und nennt 28 Literaturangaben. Gefördert von: National Key R&D Program of China, Tsinghua University-Toyota Joint Research Center for AI Technology of Automated Vehicle, National Natural Science Foundation of China.
Abstract (Originalsprache)
Though policy evaluation error profoundly affects the direction of policy optimization and the convergence property, it is usually ignored in policy iteration methods. This work incorporates the practical inexact policy evaluation into a simultaneous policy update paradigm to reach the Nash equilibrium of the nonlinear zero-sum games. In the proposed algorithm, the restriction of precise policy evaluation is removed by bounded evaluation error characterized by Hamiltonian without sacrificing convergence guarantees. By exploiting Fréchet differential, the practical iterative process of value function with estimation error is converted into the Newton's method with variable steps, which are inversely proportional to evaluation errors. Accordingly, we construct a monotone scalar sequence that shares the same Newton's method with the value sequence to bound the error of the value function, which enjoys an exponential convergence rate. Numerical results show its convergence in affine systems, and the potential to cope with general nonlinear plants.
Förderung
- National Key R&D Program of China
- Tsinghua University-Toyota Joint Research Center for AI Technology of Automated Vehicle
- National Natural Science Foundation of China
Zitieren
APA
Li, J., Li, S. E., Duan, J., Lyu, Y., Zou, W., Guan, Y., & Yin, Y. (2024). Relaxed Policy Iteration Algorithm for Nonlinear Zero-Sum Games With Application to H-Infinity Control. IEEE Transactions on Automatic Control, 69(1), 426-433. https://doi.org/10.1109/tac.2023.3266277
Vancouver
Li J, Li SE, Duan J, Lyu Y, Zou W, Guan Y, et al. Relaxed Policy Iteration Algorithm for Nonlinear Zero-Sum Games With Application to H-Infinity Control. IEEE Transactions on Automatic Control. 2024;69(1):426-433. doi:10.1109/tac.2023.3266277
BibTeX
@article{Li2024,
author = {Li, Jie and Li, Shengbo Eben and Duan, Jingliang and Lyu, Yao and Zou, Wenjun and Guan, Yang and Yin, Yuming},
title = {Relaxed Policy Iteration Algorithm for Nonlinear Zero-Sum Games With Application to H-Infinity Control},
journal = {IEEE Transactions on Automatic Control},
year = {2024},
volume = {69},
number = {1},
pages = {426--433},
doi = {10.1109/tac.2023.3266277}
}
EUVIMED