FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving
IEEE Transactions on Intelligent Transportation Systems · 2023-11 · 24 (11) · 12154-12165
DOI: 10.1109/tits.2023.3287557 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Intelligent Transportation Systems, 2023-11. Verfasst von Lei Zhang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 39 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Haihe Laboratory of Information Technology Application Innovation, Natural Science Foundation of Tianjin.
Abstract (Originalsprache)
As a key task in autonomous driving, 3D object detection based on LiDAR-camera fusion is expected to achieve more robust results by the complementarity of the two sensors. However, LiDAR-camera fusion is non-trivial. An existing problem for this type of detector is that the scale and receptive field of LiDAR point features and image features are not matched, leading to information deficiency or redundancy in fusion. This paper proposes a Point-based Pyramid Attention Fusion (PPAF) module for LiDAR-camera fusion to solve the problem. The PPAF module learns corresponding image features of LiDAR points with a matched scale based on the image feature pyramid and attention mechanism for a better effect of fusion. Furthermore, based on the PPAF module, a new LiDAR-camera fusion-based 3D object detector named FS-Net is proposed, a two-stage detector with LiDAR voxel-based RPN and refinement network based on enriched LiDAR-camera features. Experiments on two public datasets demonstrate the effectiveness of our approach.
Förderung
- National Natural Science Foundation of China
- Haihe Laboratory of Information Technology Application Innovation
- Natural Science Foundation of Tianjin
- Opening the Fundamental Research Funds for the Central Universities Jilin University
- Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing, China
Zitieren
APA
Zhang, L., Li, X., Tang, K., Jiang, Y., Yang, L., Zhang, Y., & Chen, X. (2023). FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving. IEEE Transactions on Intelligent Transportation Systems, 24(11), 12154-12165. https://doi.org/10.1109/tits.2023.3287557
Vancouver
Zhang L, Li X, Tang K, Jiang Y, Yang L, Zhang Y, et al. FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving. IEEE Transactions on Intelligent Transportation Systems. 2023;24(11):12154-12165. doi:10.1109/tits.2023.3287557
BibTeX
@article{Zhang2023,
author = {Zhang, Lei and Li, Xu and Tang, Kaichen and Jiang, Yunzhe and Yang, Liu and Zhang, Yonggang and Chen, Xianyi},
title = {FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving},
journal = {IEEE Transactions on Intelligent Transportation Systems},
year = {2023},
volume = {24},
number = {11},
pages = {12154--12165},
doi = {10.1109/tits.2023.3287557}
}
EUVIMED