RCRFNet: Enhancing Object Detection with Self-Supervised Radar–Camera Fusion and Open-Set Recognition
Sensors · 24.07.2024 · 24 (15) · 4803
DOI: 10.3390/s24154803 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 24.07.2024. Verfasst von Minwei Chen und 3 weiteren Autor:innen. Beteiligte Einrichtung: Shanghai Key Laboratory of Intelligent Sensing and Recognition, Shanghai Jiao Tong University, Shanghai 200240, China. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the National Natural Science Foundation of China.
Abstract (Originalsprache)
Robust object detection in complex environments, poor visual conditions, and open scenarios presents significant technical challenges in autonomous driving. These challenges necessitate the development of advanced fusion methods for millimeter-wave (mmWave) radar point cloud data and visual images. To address these issues, this paper proposes a radar–camera robust fusion network (RCRFNet), which leverages self-supervised learning and open-set recognition to effectively utilise the complementary information from both sensors. Specifically, the network uses matched radar–camera data through a frustum association approach to generate self-supervised signals, enhancing network training. The integration of global and local depth consistencies between radar point clouds and visual images, along with image features, helps construct object class confidence levels for detecting unknown targets. Additionally, these techniques are combined with a multi-layer feature extraction backbone and a multimodal feature detection head to achieve robust object detection. Experiments on the nuScenes public dataset demonstrate that RCRFNet outperforms state-of-the-art (SOTA) methods, particularly in conditions of low visual visibility and when detecting unknown class objects.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Shanghai Key Laboratory of Intelligent Sensing and Recognition, Shanghai Jiao Tong University, Shanghai 200240, China.
- Center of Digital Innovation, Tongji University, Shanghai 200092, China.
ORCID-Profile: Zenghui Zhang, Weiwei Guo
Förderung
- the National Natural Science Foundation of China
Zitieren
APA
Chen, M., Liu, Y., Zhang, Z., & Guo, W. (2024). RCRFNet: Enhancing Object Detection with Self-Supervised Radar–Camera Fusion and Open-Set Recognition. Sensors, 24(15), 4803. https://doi.org/10.3390/s24154803
Vancouver
Chen M, Liu Y, Zhang Z, Guo W. RCRFNet: Enhancing Object Detection with Self-Supervised Radar–Camera Fusion and Open-Set Recognition. Sensors. 2024;24(15):4803. doi:10.3390/s24154803
BibTeX
@article{Chen2024,
author = {Chen, Minwei and Liu, Yajun and Zhang, Zenghui and Guo, Weiwei},
title = {RCRFNet: Enhancing Object Detection with Self-Supervised Radar–Camera Fusion and Open-Set Recognition},
journal = {Sensors},
year = {2024},
volume = {24},
number = {15},
pages = {4803},
doi = {10.3390/s24154803}
}
EUVIMED