EUVIMEDEUVIMED entdecken
Crossref

Online Off-Policy Reinforcement Learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks

Liao Zhu, Qinglai Wei, Ping Guo

IEEE Transactions on Systems, Man, and Cybernetics: Systems · 2024-08 · 54 (8) · 5112-5122

DOI: 10.1109/tsmc.2024.3392756 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2024-08. Verfasst von Liao Zhu und 2 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 46 Literaturangaben. Gefördert von: National Key Reseanch and Development Program of China, National Natural Science Foundation of China, Science and Technology Development Fund, Macau, SAR.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this article, a real-time online off-policy reinforcement learning (RL) method is developed for the optimal control problem of unknown continuous-time nonlinear systems. First, by applying the temporal difference technique to the iterative procedure of off-policy RL, the iterative value function and the iterative policy input can be learned in real-time online. It is proven that the fitting error of neural network (NN) weights is exponentially convergent in each iteration. Second, a model-free Hamilton–Jacobi–Bellman equation (MF-HJBE) is deduced by taking the limit of the iterative procedure of off-policy RL. In this manner, it not only eliminates system dynamics in the classical HJBE, but also vanishes the iteration index. By applying temporal difference to the MF-HJBE, a real-time online tuning rule is designed to learn the optimal value function and the optimal policy input. It is proven that the fitting error of NN weights caused by the real-time online tuning rule is exponentially convergent. Note that the two online tuning rules, the iterative one and the real-time one, use only current and previous state data extracted from system trajectories. Meanwhile, it is proven using the Lyapunov’s direct method that the system solution is uniformly ultimately bounded. Finally, simulation results demonstrate the validity of the proffered method.

Förderung

  • National Key Reseanch and Development Program of China
  • National Natural Science Foundation of China
  • Science and Technology Development Fund, Macau, SAR

Zitieren

APA

Zhu, L., Wei, Q., & Guo, P. (2024). Online Off-Policy Reinforcement Learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 54(8), 5112-5122. https://doi.org/10.1109/tsmc.2024.3392756

Vancouver

Zhu L, Wei Q, Guo P. Online Off-Policy Reinforcement Learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2024;54(8):5112-5122. doi:10.1109/tsmc.2024.3392756

BibTeX
@article{Zhu2024,
  author = {Zhu, Liao and Wei, Qinglai and Guo, Ping},
  title = {Online Off-Policy Reinforcement Learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks},
  journal = {IEEE Transactions on Systems, Man, and Cybernetics: Systems},
  year = {2024},
  volume = {54},
  number = {8},
  pages = {5112--5122},
  doi = {10.1109/tsmc.2024.3392756}
}
Kostenlos starten →