EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Learnable fusion mechanisms for multimodal object detection in autonomous vehicles

Yahya Massoud, Robert Laganiere

IET Computer Vision · 2024-06 · 18 (4) · 499-511

DOI: 10.1049/cvi2.12259 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IET Computer Vision, online erschienen am 15.03.2024. Verfasst von Yahya Massoud und 1 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Perception systems in autonomous vehicles need to accurately detect and classify objects within their surrounding environments. Numerous types of sensors are deployed on these vehicles, and the combination of such multimodal data streams can significantly boost performance. The authors introduce a novel sensor fusion framework using deep convolutional neural networks. The framework employs both camera and LiDAR sensors in a multimodal, multiview configuration. The authors leverage both data types by introducing two new innovative fusion mechanisms: element‐wise multiplication and multimodal factorised bilinear pooling. The methods improve the bird's eye view moderate average precision score by +4.97% and +8.35% on the KITTI dataset when compared to traditional fusion operators like element‐wise addition and feature map concatenation. An in‐depth analysis of key design choices impacting performance, such as data augmentation, multi‐task learning, and convolutional architecture design is offered. The study aims to pave the way for the development of more robust multimodal machine vision systems. The authors conclude the paper with qualitative results, discussing both successful and problematic cases, along with potential ways to mitigate the latter.

Zitieren

APA

Massoud, Y., & Laganiere, R. (2024). Learnable fusion mechanisms for multimodal object detection in autonomous vehicles. IET Computer Vision, 18(4), 499-511. https://doi.org/10.1049/cvi2.12259

Vancouver

Massoud Y, Laganiere R. Learnable fusion mechanisms for multimodal object detection in autonomous vehicles. IET Computer Vision. 2024;18(4):499-511. doi:10.1049/cvi2.12259

BibTeX
@article{Massoud2024,
  author = {Massoud, Yahya and Laganiere, Robert},
  title = {Learnable fusion mechanisms for multimodal object detection in autonomous vehicles},
  journal = {IET Computer Vision},
  year = {2024},
  volume = {18},
  number = {4},
  pages = {499--511},
  doi = {10.1049/cvi2.12259}
}
Kostenlos starten →