EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection

Rui Wan iD, Tianyun Zhao, Wei Zhao iD

Sensors · 17.03.2023 · 23 (6) · 3229

DOI: 10.3390/s23063229 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 17.03.2023. Verfasst von Rui Wan und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Automation, Northwestern Polytechnical University, Xi'an 710129, China. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In autonomous driving, 3D object detection based on multi-modal data has become an indispensable perceptual approach when facing complex environments around the vehicle. During multi-modal detection, LiDAR and a camera are simultaneously applied for capturing and modeling. However, due to the intrinsic discrepancies between the LiDAR point and camera image, the fusion of the data for object detection encounters a series of problems, which results in most multi-modal detection methods performing worse than LiDAR-only methods. In this investigation, we propose a method named PTA-Det to improve the performance of multi-modal detection. Accompanied by PTA-Det, a Pseudo Point Cloud Generation Network is proposed, which can represent the textural and semantic features of keypoints in the image by pseudo points. Thereafter, through a transformer-based Point Fusion Transition (PFT) module, the features of LiDAR points and pseudo points from an image can be deeply fused under a unified point-based form. The combination of these modules can overcome the main obstacle of cross-modal feature fusion and achieves a complementary and discriminative representation for proposal generation. Extensive experiments on KITTI dataset support the effectiveness of PTA-Det, achieving a mAP (mean average precision) of 77.88% on the car category with relatively few LiDAR input points.

Schlagwörter

Stichwörter der Autor:innen

3D object detectionautomatic drivingmulti-modal fusionpoint cloud

Autor:innen und Institutionen

  • School of Automation, Northwestern Polytechnical University, Xi'an 710129, China.
  • Institute of Photonics & Photon Technology, Northwest University, Xi'an 710127, China.

ORCID-Profile: Rui Wan, Wei Zhao

Zitieren

APA

Wan, R., Zhao, T., & Zhao, W. (2023). PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection. Sensors, 23(6), 3229. https://doi.org/10.3390/s23063229

Vancouver

Wan R, Zhao T, Zhao W. PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection. Sensors. 2023;23(6):3229. doi:10.3390/s23063229

BibTeX
@article{Wan2023,
  author = {Wan, Rui and Zhao, Tianyun and Zhao, Wei},
  title = {PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {6},
  pages = {3229},
  doi = {10.3390/s23063229}
}
Kostenlos starten →