PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection
Sensors · 17.03.2023 · 23 (6) · 3229
DOI: 10.3390/s23063229 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 17.03.2023. Verfasst von Rui Wan und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Automation, Northwestern Polytechnical University, Xi'an 710129, China. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In autonomous driving, 3D object detection based on multi-modal data has become an indispensable perceptual approach when facing complex environments around the vehicle. During multi-modal detection, LiDAR and a camera are simultaneously applied for capturing and modeling. However, due to the intrinsic discrepancies between the LiDAR point and camera image, the fusion of the data for object detection encounters a series of problems, which results in most multi-modal detection methods performing worse than LiDAR-only methods. In this investigation, we propose a method named PTA-Det to improve the performance of multi-modal detection. Accompanied by PTA-Det, a Pseudo Point Cloud Generation Network is proposed, which can represent the textural and semantic features of keypoints in the image by pseudo points. Thereafter, through a transformer-based Point Fusion Transition (PFT) module, the features of LiDAR points and pseudo points from an image can be deeply fused under a unified point-based form. The combination of these modules can overcome the main obstacle of cross-modal feature fusion and achieves a complementary and discriminative representation for proposal generation. Extensive experiments on KITTI dataset support the effectiveness of PTA-Det, achieving a mAP (mean average precision) of 77.88% on the car category with relatively few LiDAR input points.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Automation, Northwestern Polytechnical University, Xi'an 710129, China.
- Institute of Photonics & Photon Technology, Northwest University, Xi'an 710127, China.
Zitieren
APA
Wan, R., Zhao, T., & Zhao, W. (2023). PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection. Sensors, 23(6), 3229. https://doi.org/10.3390/s23063229
Vancouver
Wan R, Zhao T, Zhao W. PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection. Sensors. 2023;23(6):3229. doi:10.3390/s23063229
BibTeX
@article{Wan2023,
author = {Wan, Rui and Zhao, Tianyun and Zhao, Wei},
title = {PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection},
journal = {Sensors},
year = {2023},
volume = {23},
number = {6},
pages = {3229},
doi = {10.3390/s23063229}
}
EUVIMED