X-Align++: cross-modal cross-view alignment for Bird’s-eye-view segmentation
Machine Vision and Applications · 2023-07 · 34 (4)
DOI: 10.1007/s00138-023-01400-7 ↗Kurz gefasst
Fachartikel in der Zeitschrift Machine Vision and Applications, online erschienen am 16.05.2023. Verfasst von Shubhankar Borse und 6 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 71 Literaturangaben.
Abstract (Originalsprache)
Bird’s-eye-view (BEV) grid is a typical representation of the perception of road components, e.g., drivable area, in autonomous driving. Most existing approaches rely on cameras only to perform segmentation in BEV space, which is fundamentally constrained by the absence of reliable depth information. The latest works leverage both camera and LiDAR modalities but suboptimally fuse their features using simple, concatenation-based mechanisms. In this paper, we address these problems by enhancing the alignment of the unimodal features in order to aid feature fusion, as well as enhancing the alignment between the cameras’ perspective view (PV) and BEV representations. We propose X-Align, a novel end-to-end cross-modal and cross-view learning framework for BEV segmentation consisting of the following components: (i) a novel Cross-Modal Feature Alignment (X-FA) loss, (ii) an attention-based Cross-Modal Feature Fusion (X-FF) module to align multi-modal BEV features implicitly, and (iii) an auxiliary PV segmentation branch with Cross-View Segmentation Alignment (X-SA) losses to improve the PV-to-BEV transformation. We evaluate our proposed method across two commonly used benchmark datasets, i.e., nuScenes and KITTI-360. Notably, X-Align significantly outperforms the state-of-the-art by 3 absolute mIoU points on nuScenes. We also provide extensive ablation studies to demonstrate the effectiveness of the individual components.
Zitieren
APA
Borse, S., Klingner, M., Ravi, V., Cai, H., Almuzairee, A., Yogamani, S., & Porikli, F. (2023). X-Align++: cross-modal cross-view alignment for Bird’s-eye-view segmentation. Machine Vision and Applications, 34(4). https://doi.org/10.1007/s00138-023-01400-7
Vancouver
Borse S, Klingner M, Ravi V, Cai H, Almuzairee A, Yogamani S, et al. X-Align++: cross-modal cross-view alignment for Bird’s-eye-view segmentation. Machine Vision and Applications. 2023;34(4). doi:10.1007/s00138-023-01400-7
BibTeX
@article{Borse2023,
author = {Borse, Shubhankar and Klingner, Marvin and Ravi, Varun and Cai, Hong and Almuzairee, Abdulaziz and Yogamani, Senthil and Porikli, Fatih},
title = {X-Align++: cross-modal cross-view alignment for Bird’s-eye-view segmentation},
journal = {Machine Vision and Applications},
year = {2023},
volume = {34},
number = {4},
doi = {10.1007/s00138-023-01400-7}
}
EUVIMED