Camera–Radar Fusion with Modality Interaction and Radar Gaussian Expansion for 3D Object Detection
Cyborg and Bionic Systems · 2024-01 · 5
DOI: 10.34133/cbsystems.0079 ↗Kurz gefasst
Fachartikel in der Zeitschrift Cyborg and Bionic Systems, online erschienen am 30.01.2024. Verfasst von Xiang Liu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Artificial Intelligence, Shanghai University, Shanghai, China. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Shanghai Sailing Program, National Natural Science Foundation of China.
Abstract (Originalsprache)
The fusion of millimeter-wave radar and camera modalities is crucial for improving the accuracy and completeness of 3-dimensional (3D) object detection. Most existing methods extract features from each modality separately and conduct fusion with specifically designed modules, potentially resulting in information loss during modality transformation. To address this issue, we propose a novel framework for 3D object detection that iteratively updates radar and camera features through an interaction module. This module serves a dual purpose by facilitating the fusion of multi-modal data while preserving the original features. Specifically, radar and image features are sampled and aggregated with a set of sparse 3D object queries, while retaining the integrity of the original radar features to prevent information loss. Additionally, an innovative radar augmentation technique named Radar Gaussian Expansion is proposed. This module allocates radar measurements within each voxel to neighboring ones as a Gaussian distribution, reducing association errors during projection and enhancing detection accuracy. Our proposed framework offers a comprehensive solution to the fusion of radar and camera data, ultimately leading to heightened accuracy and completeness in 3D object detection processes. On the nuScenes test benchmark, our camera–radar fusion method achieves state-of-the-art 3D object detection results with a 41.6% mean average precision and 52.5% nuScenes detection score.
Autor:innen und Institutionen
- Institute of Artificial Intelligence, Shanghai University, Shanghai, China.
- School of Future Technology, Shanghai University, Shanghai, China.
- State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing, China.
ORCID-Profile: Zhenglin Li
Förderung
- Shanghai Sailing Program
- National Natural Science Foundation of China
Zitieren
APA
Liu, X., Li, Z., Zhou, Y., Peng, Y., & Luo, J. (2024). Camera–Radar Fusion with Modality Interaction and Radar Gaussian Expansion for 3D Object Detection. Cyborg and Bionic Systems, 5. https://doi.org/10.34133/cbsystems.0079
Vancouver
Liu X, Li Z, Zhou Y, Peng Y, Luo J. Camera–Radar Fusion with Modality Interaction and Radar Gaussian Expansion for 3D Object Detection. Cyborg and Bionic Systems. 2024;5. doi:10.34133/cbsystems.0079
BibTeX
@article{Liu2024,
author = {Liu, Xiang and Li, Zhenglin and Zhou, Yang and Peng, Yan and Luo, Jun},
title = {Camera–Radar Fusion with Modality Interaction and Radar Gaussian Expansion for 3D Object Detection},
journal = {Cyborg and Bionic Systems},
year = {2024},
volume = {5},
doi = {10.34133/cbsystems.0079}
}
EUVIMED