NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (5) · 4542-4550
DOI: 10.1609/aaai.v38i5.28253 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Tianwen Qian und 4 weiteren Autor:innen. Die Arbeit wurde bisher 196-mal zitiert (Crossref).
Abstract (Originalsprache)
We introduce a novel visual question answering (VQA) task in the context of autonomous driving, aiming to answer natural language questions based on street-view clues. Compared to traditional VQA tasks, VQA in autonomous driving scenario presents more challenges. Firstly, the raw visual data are multi-modal, including images and point clouds captured by camera and LiDAR, respectively. Secondly, the data are multi-frame due to the continuous, real-time acquisition. Thirdly, the outdoor scenes exhibit both moving foreground and static background. Existing VQA benchmarks fail to adequately address these complexities. To bridge this gap, we propose NuScenes-QA, the first benchmark for VQA in the autonomous driving scenario, encompassing 34K visual scenes and 460K question-answer pairs. Specifically, we leverage existing 3D detection annotations to generate scene graphs and design question templates manually. Subsequently, the question-answer pairs are generated programmatically based on these templates. Comprehensive statistics prove that our NuScenes-QA is a balanced large-scale benchmark with diverse question formats. Built upon it, we develop a series of baselines that employ advanced 3D detection and VQA techniques. Our extensive experiments highlight the challenges posed by this new task. Codes and dataset are available at https://github.com/qiantianwen/NuScenes-QA.
Zitieren
APA
Qian, T., Chen, J., Zhuo, L., Jiao, Y., & Jiang, Y. (2024). NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario. Proceedings of the AAAI Conference on Artificial Intelligence, 38(5), 4542-4550. https://doi.org/10.1609/aaai.v38i5.28253
Vancouver
Qian T, Chen J, Zhuo L, Jiao Y, Jiang Y. NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(5):4542-4550. doi:10.1609/aaai.v38i5.28253
BibTeX
@article{Qian2024,
author = {Qian, Tianwen and Chen, Jingjing and Zhuo, Linhai and Jiao, Yang and Jiang, Yu-Gang},
title = {NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {5},
pages = {4542--4550},
doi = {10.1609/aaai.v38i5.28253}
}
EUVIMED