TL-4DRCF: A Two-Level 4-D Radar–Camera Fusion Method for Object Detection in Adverse Weather
IEEE Sensors Journal · 15.05.2024 · 24 (10) · 16408-16418
DOI: 10.1109/jsen.2024.3382669 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Sensors Journal, erschienen am 15.05.2024. Verfasst von Haoyi Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 46 Literaturangaben. Gefördert von: Development of the Marine Economy in Guangdong Province, Cooperative Scientific Research Project of the “Chunhui Program” of the Ministry of Education of China, Ministry of Education University-Industry Collaborative Education Program.
Abstract (Originalsprache)
In autonomous driving systems, cameras and light detection and ranging (LiDAR) are two common sensors for object detection. However, both sensors can be severely affected by adverse weather. With the development of radar technology, the emergence of the 4-D radar gives a more robust solution for sensor fusion strategies in 3-D object detection tasks. This study proposes a two-level 4-D radar and camera fusion model called TL-4DRCF, which performs a two-level fusion of the 4-D radar and camera information at the data and feature levels. In the data-level (DL) fusion stage, the radar point cloud is projected onto the image and fed as additional information to the image into the EarlyFusion-Net (EF-Net), which is the network designed for simultaneous extraction of point cloud and image features. In the feature-level fusion stage, the radar–camera alignment (RCA) module is proposed to accurately correlate point cloud voxels and pixel-level image features while consuming less inference time. The correlated features are used to predict the class and location of the object through a standard 3-D detection framework. The proposed TL-4DRCF was validated on the View-of-Delft (VoD) dataset and the VoD-Fog dataset performed by artificial fog processing. The experimental results show that the proposed model outperforms the baseline method PointPillars on the VoD dataset by 3.8% mAP and the LiDAR–camera-based method MVX-Net in the driving corridor area of the VoD-Fog dataset by 0.39% mAP.
Förderung
- Development of the Marine Economy in Guangdong Province
- Cooperative Scientific Research Project of the “Chunhui Program” of the Ministry of Education of China
- Ministry of Education University-Industry Collaborative Education Program
Zitieren
APA
Zhang, H., Wu, K., Chen, R., Wu, Z., Zhong, Y., & Li, W. (2024). TL-4DRCF: A Two-Level 4-D Radar–Camera Fusion Method for Object Detection in Adverse Weather. IEEE Sensors Journal, 24(10), 16408-16418. https://doi.org/10.1109/jsen.2024.3382669
Vancouver
Zhang H, Wu K, Chen R, Wu Z, Zhong Y, Li W. TL-4DRCF: A Two-Level 4-D Radar–Camera Fusion Method for Object Detection in Adverse Weather. IEEE Sensors Journal. 2024;24(10):16408-16418. doi:10.1109/jsen.2024.3382669
BibTeX
@article{Zhang2024,
author = {Zhang, Haoyi and Wu, Kai and Chen, Rongkang and Wu, Zihao and Zhong, Yong and Li, Weihua},
title = {TL-4DRCF: A Two-Level 4-D Radar–Camera Fusion Method for Object Detection in Adverse Weather},
journal = {IEEE Sensors Journal},
year = {2024},
volume = {24},
number = {10},
pages = {16408--16418},
doi = {10.1109/jsen.2024.3382669}
}
EUVIMED