Dual-Critic Proximal Policy Optimization for Trajectory Tracking of Autonomous Underwater Vehicle Under Time-Varying Dynamics
IEEE Journal of Oceanic Engineering · 2026-07 · 51 (3) · 1899-1912
DOI: 10.1109/joe.2026.3692007 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Journal of Oceanic Engineering, 2026-07. Verfasst von Ye Yuan und 4 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
The autonomous underwater vehicle (AUV) serves as an ideal platform for replacing human operators in high-risk underwater operations, where high-precision trajectory tracking control is crucial for ensuring mission effectiveness. However, the time-varying dynamics caused by the complexity of the underwater environment present significant challenges to achieving high-precision trajectory tracking for AUVs. To address this challenge, a model-assisted deep reinforcement learning-based trajectory tracking method for an AUV with time-varying dynamics is presented in this article. Specifically, a novel state space is proposed, which replaces the absolute position of the agent with the relative position. This representation provides a more comprehensive description of the relationship between the AUV and the reference trajectory. The policy trained based on this state space can be easily transferred and applied to a different type of reference trajectory. Furthermore, by decomposing the reward function into three components, the sparse reward issue is effectively mitigated. The smoothness term within the reward function effectively suppresses abrupt changes in control commands, ensuring that the AUV maintains smooth navigation even under time-varying dynamics. Finally, an improved proximal policy optimization algorithm is proposed. The algorithm includes two critics that separately evaluate the value functions of actions under time-varying and static dynamics, and adaptively adjust their weights based on the intensity of the time-varying dynamics, effectively enhancing the policy’s disturbance rejection capability and robustness. Experimental results demonstrate that the proposed method maintains high-precision trajectory tracking across various time-varying dynamics, significantly outperforming other algorithms and validating its effectiveness.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Yuan, Y., Chen, C., Yang, D., Zhang, W., & Wang, Y. (2026). Dual-Critic Proximal Policy Optimization for Trajectory Tracking of Autonomous Underwater Vehicle Under Time-Varying Dynamics. IEEE Journal of Oceanic Engineering, 51(3), 1899-1912. https://doi.org/10.1109/joe.2026.3692007
Vancouver
Yuan Y, Chen C, Yang D, Zhang W, Wang Y. Dual-Critic Proximal Policy Optimization for Trajectory Tracking of Autonomous Underwater Vehicle Under Time-Varying Dynamics. IEEE Journal of Oceanic Engineering. 2026;51(3):1899-1912. doi:10.1109/joe.2026.3692007
BibTeX
@article{Yuan2026,
author = {Yuan, Ye and Chen, Chaoyang and Yang, Dan and Zhang, Weidong and Wang, Yaonan},
title = {Dual-Critic Proximal Policy Optimization for Trajectory Tracking of Autonomous Underwater Vehicle Under Time-Varying Dynamics},
journal = {IEEE Journal of Oceanic Engineering},
year = {2026},
volume = {51},
number = {3},
pages = {1899--1912},
doi = {10.1109/joe.2026.3692007}
}
EUVIMED