EUVIMEDEUVIMED entdecken
Open Access

Camera–Radar Fusion with Modality Interaction and Radar Gaussian Expansion for 3D Object Detection

Xiang Liu, Zhenglin Li iD, Yang Zhou, Yan Peng, Jun Luo

Cyborg and Bionic Systems · 2024-01 · 5

DOI: 10.34133/cbsystems.0079 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cyborg and Bionic Systems, online erschienen am 30.01.2024. Verfasst von Xiang Liu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Artificial Intelligence, Shanghai University, Shanghai, China. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Shanghai Sailing Program, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The fusion of millimeter-wave radar and camera modalities is crucial for improving the accuracy and completeness of 3-dimensional (3D) object detection. Most existing methods extract features from each modality separately and conduct fusion with specifically designed modules, potentially resulting in information loss during modality transformation. To address this issue, we propose a novel framework for 3D object detection that iteratively updates radar and camera features through an interaction module. This module serves a dual purpose by facilitating the fusion of multi-modal data while preserving the original features. Specifically, radar and image features are sampled and aggregated with a set of sparse 3D object queries, while retaining the integrity of the original radar features to prevent information loss. Additionally, an innovative radar augmentation technique named Radar Gaussian Expansion is proposed. This module allocates radar measurements within each voxel to neighboring ones as a Gaussian distribution, reducing association errors during projection and enhancing detection accuracy. Our proposed framework offers a comprehensive solution to the fusion of radar and camera data, ultimately leading to heightened accuracy and completeness in 3D object detection processes. On the nuScenes test benchmark, our camera–radar fusion method achieves state-of-the-art 3D object detection results with a 41.6% mean average precision and 52.5% nuScenes detection score.

Autor:innen und Institutionen

  • Institute of Artificial Intelligence, Shanghai University, Shanghai, China.
  • School of Future Technology, Shanghai University, Shanghai, China.
  • State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing, China.

ORCID-Profile: Zhenglin Li

Förderung

  • Shanghai Sailing Program
  • National Natural Science Foundation of China

Zitieren

APA

Liu, X., Li, Z., Zhou, Y., Peng, Y., & Luo, J. (2024). Camera–Radar Fusion with Modality Interaction and Radar Gaussian Expansion for 3D Object Detection. Cyborg and Bionic Systems, 5. https://doi.org/10.34133/cbsystems.0079

Vancouver

Liu X, Li Z, Zhou Y, Peng Y, Luo J. Camera–Radar Fusion with Modality Interaction and Radar Gaussian Expansion for 3D Object Detection. Cyborg and Bionic Systems. 2024;5. doi:10.34133/cbsystems.0079

BibTeX
@article{Liu2024,
  author = {Liu, Xiang and Li, Zhenglin and Zhou, Yang and Peng, Yan and Luo, Jun},
  title = {Camera–Radar Fusion with Modality Interaction and Radar Gaussian Expansion for 3D Object Detection},
  journal = {Cyborg and Bionic Systems},
  year = {2024},
  volume = {5},
  doi = {10.34133/cbsystems.0079}
}
Kostenlos starten →