EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

HDAO: A Hierarchical Curiosity-Driven Reinforcement Learning Approach for AUV Dynamic Obstacle Avoidance

Huazheng Du, Qian Liu, Xu Liu, Na Xia

Journal of Marine Science and Engineering · 14.04.2026 · 14 (8) · 720

DOI: 10.3390/jmse14080720 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Marine Science and Engineering, online erschienen am 14.04.2026. Verfasst von Huazheng Du und 3 weiteren Autor:innen. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Autonomous obstacle avoidance is a critical capability for Autonomous Underwater Vehicles (AUVs) to operate safely in dynamic and uncertain marine environments. Traditional AUV control methods rely on precise physical modeling and preset rules, yet they struggle to adapt to multiple sources of uncertainty, such as random initial states, dynamic obstacles, and varying currents. In recent years, deep reinforcement learning has provided a new avenue for data-driven adaptive policy learning. However, it remains insufficient for handling long-horizon tasks with sparse rewards. While hierarchical reinforcement learning can mitigate reward sparsity through temporal abstraction, it often faces challenges including exploration–exploitation imbalance, slow global convergence, and insufficient safety guarantees. Furthermore, most existing studies neglect dynamic environmental disturbances and task continuity, which further limits the practical application of these algorithms. To address these challenges, this paper proposes a hierarchical curiosity-driven AUV obstacle avoidance algorithm (HDAO), designed for autonomous obstacle avoidance in dynamic and uncertain underwater environments. The core design of HDAO incorporates several key innovations. Firstly, it introduces a Collision Threat Index for dynamic obstacles, which enables explicit risk perception and quantifies collision threats, thereby enhancing the policy’s generalization and robustness. Secondly, a task-decoupled hierarchical architecture is employed to synergistically optimize global path planning and local obstacle avoidance behaviors. This approach effectively manages long-horizon navigation tasks while alleviating high-dimensional training pressure. Finally, a novel reward mechanism is designed by integrating hierarchical active exploration with curiosity-driven passive exploration. This mechanism effectively incentivizes the agent to explore unvisited areas under sparse reward conditions and dynamically balances exploration and exploitation. Experimental results demonstrate that HDAO significantly outperforms existing methods in terms of obstacle avoidance success rate, training convergence speed and robustness against external disturbances.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Du, H., Liu, Q., Liu, X., & Xia, N. (2026). HDAO: A Hierarchical Curiosity-Driven Reinforcement Learning Approach for AUV Dynamic Obstacle Avoidance. Journal of Marine Science and Engineering, 14(8), 720. https://doi.org/10.3390/jmse14080720

Vancouver

Du H, Liu Q, Liu X, Xia N. HDAO: A Hierarchical Curiosity-Driven Reinforcement Learning Approach for AUV Dynamic Obstacle Avoidance. Journal of Marine Science and Engineering. 2026;14(8):720. doi:10.3390/jmse14080720

BibTeX
@article{Du2026,
  author = {Du, Huazheng and Liu, Qian and Liu, Xu and Xia, Na},
  title = {HDAO: A Hierarchical Curiosity-Driven Reinforcement Learning Approach for AUV Dynamic Obstacle Avoidance},
  journal = {Journal of Marine Science and Engineering},
  year = {2026},
  volume = {14},
  number = {8},
  pages = {720},
  doi = {10.3390/jmse14080720}
}
Kostenlos starten →