A Deep-Learning Method Based on the Multistage Fusion of Radar and Camera in UAV Obstacle Avoidance
IEEE Transactions on Aerospace and Electronic Systems · 2024-10 · 60 (5) · 6734-6751
DOI: 10.1109/taes.2024.3406495 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Aerospace and Electronic Systems, 2024-10. Verfasst von Xiyue Wang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 46 Literaturangaben. Gefördert von: Major Scientific and Technological Innovation Project of Shandong Province of China.
Abstract (Originalsprache)
Multisensor information fusion technology has been widely used in the perception of unmanned aerial vehicle environments. However, the perception accuracy needs to be improved in practice since multiple sensors have consistency limitations and fused data have limited utility. A deep-learning method based on the multistage fusion of millimeter-wave radar and camera is proposed in this article. In the data preprocessing stage, the radar reflection point and image pixels are fused in a Gaussian-weighted way to obtain the salient image. The salient density map of each pixel relative to the radar reflection point is calculated. Then, the threshold is set to segment the salient density map to complete visual target detection. In the detection stage based on deep learning, a network structure is designed to fuse the salient image and visual target detection images at different convolution depths. The classification, location, and size of targets are regressed by training. In the postdecoding stage, the radar reflection point is fused for local nonmaximum suppression. The nonmaximum suppression operation is started from the radar reflection point. Different from typical detection methods, the proposed method improves detection accuracy by fusing the feature information of the radar and camera in a multistage process. The experimental results demonstrated that mAP0.90 increased by 3.9% and 4.3%. For complex scenarios, mAP0.50 improved by 2.4%, mAP0.75 improved by 4.9%, and mAP0.90 improved by 6.9%, indicating that the proposed method is effective compared with the state-of-the-art model (YOLOv8).
Förderung
- Major Scientific and Technological Innovation Project of Shandong Province of China
Zitieren
APA
Wang, X., Wang, X., Zhou, Z., & Song, Y. (2024). A Deep-Learning Method Based on the Multistage Fusion of Radar and Camera in UAV Obstacle Avoidance. IEEE Transactions on Aerospace and Electronic Systems, 60(5), 6734-6751. https://doi.org/10.1109/taes.2024.3406495
Vancouver
Wang X, Wang X, Zhou Z, Song Y. A Deep-Learning Method Based on the Multistage Fusion of Radar and Camera in UAV Obstacle Avoidance. IEEE Transactions on Aerospace and Electronic Systems. 2024;60(5):6734-6751. doi:10.1109/taes.2024.3406495
BibTeX
@article{Wang2024,
author = {Wang, Xiyue and Wang, Xinsheng and Zhou, Zhiquan and Song, Yanhong},
title = {A Deep-Learning Method Based on the Multistage Fusion of Radar and Camera in UAV Obstacle Avoidance},
journal = {IEEE Transactions on Aerospace and Electronic Systems},
year = {2024},
volume = {60},
number = {5},
pages = {6734--6751},
doi = {10.1109/taes.2024.3406495}
}
EUVIMED