CLIP-BEV: A Late-Fusion Framework for Multimodal Scene Understanding Using Vision Language Models
Electronics · 13.06.2026 · 15 (12) · 2615
DOI: 10.3390/electronics15122615 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 13.06.2026. Verfasst von Fatemeh Daraee und 2 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Scene understanding is a fundamental task in autonomous driving, requiring effective integration of semantic and geometric information from heterogeneous sensors. Although vision–language models (VLMs) provide powerful semantic representations, their integration with LiDAR-based geometric perception remains challenging. This paper proposes a multimodal late-fusion framework for multi-label scene classification that combines semantic embeddings extracted from camera images using a frozen CLIP (ViT-B/32) encoder with geometric features derived from LiDAR Bird’s-Eye-View (BEV) representations. To improve multimodal compatibility, modality-specific adaptation networks are employed to refine visual and geometric features before fusion. The proposed framework was evaluated on an annotated subset of the nuScenes dataset containing synchronized camera–LiDAR samples and nine scene-level labels. Experimental results show that the proposed late-fusion architecture outperforms both unimodal and early-fusion baselines, achieving a Hamming Accuracy of 0.950, a Micro-F1 score of 0.925, and a mean Average Precision (mAP) of 0.908. Additional experiments using a CLIP-based early-fusion baseline demonstrate that the observed performance gains are primarily attributable to the proposed modality-specific refinement and late-fusion strategy rather than the visual encoder alone. These findings indicate that modality-aware late fusion of pretrained semantic representations and LiDAR geometric information provides an effective and scalable solution for multimodal perception in autonomous driving.
Zitieren
APA
Daraee, F., Mozaffari, S., & Alirezaee, S. (2026). CLIP-BEV: A Late-Fusion Framework for Multimodal Scene Understanding Using Vision Language Models. Electronics, 15(12), 2615. https://doi.org/10.3390/electronics15122615
Vancouver
Daraee F, Mozaffari S, Alirezaee S. CLIP-BEV: A Late-Fusion Framework for Multimodal Scene Understanding Using Vision Language Models. Electronics. 2026;15(12):2615. doi:10.3390/electronics15122615
BibTeX
@article{Daraee2026,
author = {Daraee, Fatemeh and Mozaffari, Saeed and Alirezaee, Shahpour},
title = {CLIP-BEV: A Late-Fusion Framework for Multimodal Scene Understanding Using Vision Language Models},
journal = {Electronics},
year = {2026},
volume = {15},
number = {12},
pages = {2615},
doi = {10.3390/electronics15122615}
}
EUVIMED