BEV-TSR: Text-Scene Retrieval in BEV Space for Autonomous Driving
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (7) · 7275-7283
DOI: 10.1609/aaai.v39i7.32782 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Tao Tang und 13 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref).
Abstract (Originalsprache)
The rapid development of the autonomous driving industry has led to a significant accumulation of autonomous driving data. Consequently, there comes a growing demand for retrieving data to provide specialized optimization. However, directly applying previous image retrieval methods faces several challenges, such as the lack of global feature representation and inadequate text retrieval ability for complex driving scenes. To address these issues, firstly, we propose the BEV-TSR framework which leverages descriptive text as an input to retrieve corresponding scenes in the Bird’s Eye View (BEV) space. Then to facilitate complex scene retrieval with extensive text descriptions, we employ a large language model (LLM) to extract the semantic features of the text inputs and incorporate knowledge graph embeddings to enhance the semantic richness of the language embedding. To achieve feature alignment between the BEV feature and language embedding, we propose Shared Cross-modal Embedding with a set of shared learnable embeddings to bridge the gap between these two modalities, and employ a caption generation task to further enhance the alignment. Furthermore, there lack of well-formed retrieval datasets for effective evaluation. To this end, we establish a multi-level retrieval dataset, nuScenes-Retrieval, based on the widely adopted nuScenes dataset. Experimental results on the multi-level nuScenes-Retrieval show that BEV-TSR achieves state-of-the-art performance, e.g., 85.78% and 87.66% top-1 accuracy on scene-to-test and text-to-scene retrieval respectively.
Zitieren
APA
Tang, T., Wei, D., Jia, Z., Gao, T., Cai, C., Hou, C., Jia, P., Zhan, K., Sun, H., JingChen, F., Zhao, Y., Liang, X., Lang, X., & Wang, Y. (2025). BEV-TSR: Text-Scene Retrieval in BEV Space for Autonomous Driving. Proceedings of the AAAI Conference on Artificial Intelligence, 39(7), 7275-7283. https://doi.org/10.1609/aaai.v39i7.32782
Vancouver
Tang T, Wei D, Jia Z, Gao T, Cai C, Hou C, et al. BEV-TSR: Text-Scene Retrieval in BEV Space for Autonomous Driving. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(7):7275-7283. doi:10.1609/aaai.v39i7.32782
BibTeX
@article{Tang2025,
author = {Tang, Tao and Wei, Dafeng and Jia, Zhengyu and Gao, Tian and Cai, Changwei and Hou, Chengkai and Jia, Peng and Zhan, Kun and Sun, Haiyang and JingChen, Fan and Zhao, Yixing and Liang, Xiaodan and Lang, Xianpeng and Wang, Yang},
title = {BEV-TSR: Text-Scene Retrieval in BEV Space for Autonomous Driving},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {7},
pages = {7275--7283},
doi = {10.1609/aaai.v39i7.32782}
}
EUVIMED