UAV Path Planning Based on the Average TD3 Algorithm With Prioritized Experience Replay
IEEE Access · 2024 · 12 · 38017-38029
DOI: 10.1109/access.2024.3375083 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2024. Verfasst von Xuqiong Luo und 3 weiteren Autor:innen. Die Arbeit wurde bisher 70-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Excellent Youth Project of Education Department of Hunan Province, National Natural Science Foundation of China, Hunan Provincial Natural Science Foundation of China.
Abstract (Originalsprache)
Path planning is one of the important components of the Unmanned Aerial Vehicle (UAV) mission, and it is also the key guarantee for the successful completion of the UAV’s mission. The traditional path planning algorithm has certain limitations and deficiencies in the complex dynamic environment. Aiming at the dynamic complex obstacle environment, this paper proposes an improved TD3 algorithm, which enables the UAV to complete the autonomous path planning through online learning and continuous trial and error. The algorithm changes the experience pool of TD3 algorithm to priority experience replay, so that the agent can distinguish the importance of empirical samples, improve the sampling efficiency of the algorithm, and reduce the training time. The average TD3 is proposed, and the average value of $Q_{1}Q_{2}$ is taken when the target value is updated to solve the problem of overestimating the $Q$ value while avoiding underestimating the $Q$ value, so that the improved algorithm has better stability and can adapt to various complex obstacle environments. A new reward function is set up, so that each step of the UAV action can receive reward feedback, which solves the problem of sparse reward in deep reinforcement learning. The experimental results show that this method can train the UAV to reach the target safely and quickly in a multi-obstacle environment. Compared with DDPG, SAC and traditional TD3, the path planning success rate of this algorithm is higher than that of the other three algorithms, and the collision rate is lower than that of the comparison algorithm, which has better path planning performance.
Förderung
- Excellent Youth Project of Education Department of Hunan Province
- National Natural Science Foundation of China
- Hunan Provincial Natural Science Foundation of China
Zitieren
APA
Luo, X., Wang, Q., Gong, H., & Tang, C. (2024). UAV Path Planning Based on the Average TD3 Algorithm With Prioritized Experience Replay. IEEE Access, 12, 38017-38029. https://doi.org/10.1109/access.2024.3375083
Vancouver
Luo X, Wang Q, Gong H, Tang C. UAV Path Planning Based on the Average TD3 Algorithm With Prioritized Experience Replay. IEEE Access. 2024;12:38017-38029. doi:10.1109/access.2024.3375083
BibTeX
@article{Luo2024,
author = {Luo, Xuqiong and Wang, Qiyuan and Gong, Hongfang and Tang, Chao},
title = {UAV Path Planning Based on the Average TD3 Algorithm With Prioritized Experience Replay},
journal = {IEEE Access},
year = {2024},
volume = {12},
pages = {38017--38029},
doi = {10.1109/access.2024.3375083}
}
EUVIMED