Point Cloud Painting for 3D Object Detection with Camera and Automotive 3+1D RADAR Fusion
Sensors · 15.02.2024 · 24 (4) · 1244
DOI: 10.3390/s24041244 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 15.02.2024. Verfasst von Santiago Montiel-Marín und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electronics, Universidad de Alcalá, 28805 Alcalá de Henares, Spain. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministerio de Ciencia e Innovación, MICIN/AEI and FEDER, MICIN/AEI.
Abstract (Originalsprache)
RADARs and cameras have been present in automotives since the advent of ADAS, as they possess complementary strengths and weaknesses but have been underlooked in the context of learning-based methods. In this work, we propose a method to perform object detection in autonomous driving based on a geometrical and sequential sensor fusion of 3+1D RADAR and semantics extracted from camera data through point cloud painting from the perspective view. To achieve this objective, we adapt PointPainting from the LiDAR and camera domains to the sensors mentioned above. We first apply YOLOv8-seg to obtain instance segmentation masks and project their results to the point cloud. As a refinement stage, we design a set of heuristic rules to minimize the propagation of errors from the segmentation to the detection stage. Our pipeline concludes by applying PointPillars as an object detection network to the painted RADAR point cloud. We validate our approach in the novel View of Delft dataset, which includes 3+1D RADAR data sequences in urban environments. Experimental results show that this fusion is also suitable for RADAR and cameras as we obtain a significant improvement over the RADAR-only baseline, increasing mAP from 41.18 to 52.67 (+27.9%).
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Electronics, Universidad de Alcalá, 28805 Alcalá de Henares, Spain.
ORCID-Profile: Santiago Montiel-Marín, Ángel Llamazares, Miguel Antunes, Pedro A Revenga, Luis M Bergasa
Förderung
- Ministerio de Ciencia e Innovación
- MICIN/AEI and FEDER
- MICIN/AEI
- European Union NextGenerationEU/PRTR
- Autonomous Community of Madrid
Zitieren
APA
Montiel-Marín, S., Llamazares, Á., Antunes, M., Revenga, P. A., & Bergasa, L. M. (2024). Point Cloud Painting for 3D Object Detection with Camera and Automotive 3+1D RADAR Fusion. Sensors, 24(4), 1244. https://doi.org/10.3390/s24041244
Vancouver
Montiel-Marín S, Llamazares Á, Antunes M, Revenga PA, Bergasa LM. Point Cloud Painting for 3D Object Detection with Camera and Automotive 3+1D RADAR Fusion. Sensors. 2024;24(4):1244. doi:10.3390/s24041244
BibTeX
@article{MontielMarn2024,
author = {Montiel-Marín, Santiago and Llamazares, Ángel and Antunes, Miguel and Revenga, Pedro A and Bergasa, Luis M},
title = {Point Cloud Painting for 3D Object Detection with Camera and Automotive 3+1D RADAR Fusion},
journal = {Sensors},
year = {2024},
volume = {24},
number = {4},
pages = {1244},
doi = {10.3390/s24041244}
}
EUVIMED