Integrating Proximal Policy Optimization with Physically Realistic Simulation for Robust Autonomous Underwater Vehicle Control
Gazi University Journal of Science Part A: Engineering and Innovation · 31.03.2026 · 13 (1) · 165-199
DOI: 10.54287/gujsa.1813751 ↗Kurz gefasst
Fachartikel in der Zeitschrift Gazi University Journal of Science Part A: Engineering and Innovation, online erschienen am 31.03.2026. Verfasst von Mahmut Mol und 1 weiteren Autor:innen. Die Arbeit nennt 29 Literaturangaben.
Abstract (Originalsprache)
This study presents the design and implementation of a reinforcement learning (RL)-based framework for the control of an autonomous underwater vehicle (AUV) directly within Unreal Engine (UE). A high-fidelity aquatic environment was created using UE’s native Water System to simulate hydrodynamic forces and buoyancy. Unlike studies assuming continuous control, this research addresses the challenge of stabilizing an AUV subject to severe discrete 'bang-bang' hardware constraints. A parallelized Proximal Policy Optimization (PPO) algorithm was employed to synthesize adaptive control policies. Comparative analysis against tuned Proportional-Integral-Derivative (PID) baselines demonstrates that the RL agent outperforms classical methods in three key metrics: (1) in longitudinal navigation, the agent learned an emergent "pulsing" strategy—mimicking Pulse-Width Modulation (PWM)—to overcome these discrete actuation constraints, reducing steady-state error compared to the Proportional-Derivative (PD) baseline; (2) in vertical depth control, the agent autonomously learned gravity compensation, settling faster than integral-based controllers while avoiding buoyancy-induced stalling; and (3) in heading control, the agent demonstrated superior dynamic handling, completing stabilization maneuvers faster than the baseline. The key architectural innovation lies in the direct integration of UE’s Learning Agents plugin, eliminating the need for external middleware. This native integration enables real-time synchronization between simulation physics and learning processes, establishing a high-fidelity platform for developing intelligent underwater control systems.
Zitieren
APA
Mol, M., & Karaarslan, A. (2026). Integrating Proximal Policy Optimization with Physically Realistic Simulation for Robust Autonomous Underwater Vehicle Control. Gazi University Journal of Science Part A: Engineering and Innovation, 13(1), 165-199. https://doi.org/10.54287/gujsa.1813751
Vancouver
Mol M, Karaarslan A. Integrating Proximal Policy Optimization with Physically Realistic Simulation for Robust Autonomous Underwater Vehicle Control. Gazi University Journal of Science Part A: Engineering and Innovation. 2026;13(1):165-199. doi:10.54287/gujsa.1813751
BibTeX
@article{Mol2026,
author = {Mol, Mahmut and Karaarslan, Ahmet},
title = {Integrating Proximal Policy Optimization with Physically Realistic Simulation for Robust Autonomous Underwater Vehicle Control},
journal = {Gazi University Journal of Science Part A: Engineering and Innovation},
year = {2026},
volume = {13},
number = {1},
pages = {165--199},
doi = {10.54287/gujsa.1813751}
}
EUVIMED