Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller
Sensors · 29.03.2023 · 23 (7) · 3572
DOI: 10.3390/s23073572 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 29.03.2023. Verfasst von Jianwen Li und 3 weiteren Autor:innen. Beteiligte Einrichtung: The School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Office of Naval Research, ORN.
Abstract (Originalsprache)
This work presents a framework that allows Unmanned Surface Vehicles (USVs) to avoid dynamic obstacles through initial training on an Unmanned Ground Vehicle (UGV) and cross-domain retraining on a USV. This is achieved by integrating a Deep Reinforcement Learning (DRL) agent that generates high-level control commands and leveraging a neural network based model predictive controller (NN-MPC) to reach target waypoints and reject disturbances. A Deep Q Network (DQN) utilized in this framework is trained in a ground environment using a Turtlebot robot and retrained in a water environment using the BREAM USV in the Gazebo simulator to avoid dynamic obstacles. The network is then validated in both simulation and real-world tests. The cross-domain learning largely decreases the training time (28%) and increases the obstacle avoidance performance (70 more reward points) compared to pure water domain training. This methodology shows that it is possible to leverage the data-rich and accessible ground environments to train DRL agent in data-poor and difficult-to-access marine environments. This will allow rapid and iterative agent development without further training due to the change in environment or vehicle dynamics.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- The School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA.
- Nvidia Corporation, Santa Clara, CA 95051, USA.
ORCID-Profile: Jianwen Li, Jalil Chavez-Galaviz, Kamyar Azizzadenesheli, Nina Mahmoudian
Förderung
- Office of Naval Research
- ORN
Zitieren
APA
Li, J., Chavez-Galaviz, J., Azizzadenesheli, K., & Mahmoudian, N. (2023). Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller. Sensors, 23(7), 3572. https://doi.org/10.3390/s23073572
Vancouver
Li J, Chavez-Galaviz J, Azizzadenesheli K, Mahmoudian N. Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller. Sensors. 2023;23(7):3572. doi:10.3390/s23073572
BibTeX
@article{Li2023,
author = {Li, Jianwen and Chavez-Galaviz, Jalil and Azizzadenesheli, Kamyar and Mahmoudian, Nina},
title = {Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller},
journal = {Sensors},
year = {2023},
volume = {23},
number = {7},
pages = {3572},
doi = {10.3390/s23073572}
}
EUVIMED