Learnable fusion mechanisms for multimodal object detection in autonomous vehicles
IET Computer Vision · 2024-06 · 18 (4) · 499-511
DOI: 10.1049/cvi2.12259 ↗Kurz gefasst
Fachartikel in der Zeitschrift IET Computer Vision, online erschienen am 15.03.2024. Verfasst von Yahya Massoud und 1 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
Perception systems in autonomous vehicles need to accurately detect and classify objects within their surrounding environments. Numerous types of sensors are deployed on these vehicles, and the combination of such multimodal data streams can significantly boost performance. The authors introduce a novel sensor fusion framework using deep convolutional neural networks. The framework employs both camera and LiDAR sensors in a multimodal, multiview configuration. The authors leverage both data types by introducing two new innovative fusion mechanisms: element‐wise multiplication and multimodal factorised bilinear pooling. The methods improve the bird's eye view moderate average precision score by +4.97% and +8.35% on the KITTI dataset when compared to traditional fusion operators like element‐wise addition and feature map concatenation. An in‐depth analysis of key design choices impacting performance, such as data augmentation, multi‐task learning, and convolutional architecture design is offered. The study aims to pave the way for the development of more robust multimodal machine vision systems. The authors conclude the paper with qualitative results, discussing both successful and problematic cases, along with potential ways to mitigate the latter.
Zitieren
APA
Massoud, Y., & Laganiere, R. (2024). Learnable fusion mechanisms for multimodal object detection in autonomous vehicles. IET Computer Vision, 18(4), 499-511. https://doi.org/10.1049/cvi2.12259
Vancouver
Massoud Y, Laganiere R. Learnable fusion mechanisms for multimodal object detection in autonomous vehicles. IET Computer Vision. 2024;18(4):499-511. doi:10.1049/cvi2.12259
BibTeX
@article{Massoud2024,
author = {Massoud, Yahya and Laganiere, Robert},
title = {Learnable fusion mechanisms for multimodal object detection in autonomous vehicles},
journal = {IET Computer Vision},
year = {2024},
volume = {18},
number = {4},
pages = {499--511},
doi = {10.1049/cvi2.12259}
}
EUVIMED