EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Multi-Ship Collision Avoidance Algorithm Using Data-Driven Multi-Agent Deep Reinforcement Learning

Yihan Niu, Feixiang Zhu, Moxuan Wei, Yifan Du, Pengyu Zhai

Journal of Marine Science and Engineering · 01.11.2023 · 11 (11) · 2101

DOI: 10.3390/jmse11112101 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Marine Science and Engineering, online erschienen am 01.11.2023. Verfasst von Yihan Niu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 37-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Liaoning Provincial Shipping Joint Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Maritime Autonomous Surface Ships (MASS) are becoming of interest to the maritime sector and are also on the agenda of the International Maritime Organization (IMO). With the boom in global maritime traffic, the number of ships is increasing rapidly. The use of intelligent technology to achieve autonomous collision avoidance is a hot issue widely discussed in the industry. In the endeavor to solve this problem, multi-ship coordinated collision avoidance has become a crucial challenge. This paper proposes a multi-ship autonomous collision avoidance decision-making algorithm by a data-driven method and adopts the Multi-agent Deep Reinforcement Learning (MADRL) framework for its design. Firstly, the overall framework of this paper and its components follow the principle of “reality as primary and simulation as supplementary”, so a real data-driven AIS (Automatic Identification System) dominates the model construction. Secondly, the agent’s observation state is determined by quantifying the hazardous area. Then, based on a full understanding of the International Regulations for Preventing Collisions at Sea (COLREGs) and the preliminary data collection, this paper combines the statistical results of the real water traffic data to guide and design the algorithm framework and selects the representative influencing factors to be designed in the collision avoidance decision-making algorithm’s reward function. Next, we train the algorithmic model using both real data and simulation data. Meanwhile, Prioritized Experience Replay (PER) is adopted to accelerate the model’s learning efficiency. Finally, 40 encounter scenarios are designed and extended to verify the algorithm performance based on the idea of the Imazu problem. The experimental results show that this algorithm can efficiently make a ship collision avoidance decision in compliance with COLREGs. Multi-agent learning through shared network policies can ensure that the agents pass beyond the safe distance in unknown environments. We can apply the trained model to the system with different numbers of agents to provide a reference for the research of autonomous collision avoidance in ships.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Liaoning Provincial Shipping Joint Fund

Zitieren

APA

Niu, Y., Zhu, F., Wei, M., Du, Y., & Zhai, P. (2023). A Multi-Ship Collision Avoidance Algorithm Using Data-Driven Multi-Agent Deep Reinforcement Learning. Journal of Marine Science and Engineering, 11(11), 2101. https://doi.org/10.3390/jmse11112101

Vancouver

Niu Y, Zhu F, Wei M, Du Y, Zhai P. A Multi-Ship Collision Avoidance Algorithm Using Data-Driven Multi-Agent Deep Reinforcement Learning. Journal of Marine Science and Engineering. 2023;11(11):2101. doi:10.3390/jmse11112101

BibTeX
@article{Niu2023,
  author = {Niu, Yihan and Zhu, Feixiang and Wei, Moxuan and Du, Yifan and Zhai, Pengyu},
  title = {A Multi-Ship Collision Avoidance Algorithm Using Data-Driven Multi-Agent Deep Reinforcement Learning},
  journal = {Journal of Marine Science and Engineering},
  year = {2023},
  volume = {11},
  number = {11},
  pages = {2101},
  doi = {10.3390/jmse11112101}
}
Kostenlos starten →