ScenePilot-4K: A Large-Scale First-Person Dataset and Benchmark for Vision-Language Models in Autonomous Driving
Communications in Transportation Research · 06.11.2026
DOI: 10.26599/commtr.2026.9640069 ↗Kurz gefasst
Fachartikel in der Zeitschrift Communications in Transportation Research, online erschienen am 06.11.2026. Verfasst von Yujin Wang und 9 weiteren Autor:innen.
Abstract (Originalsprache)
In this paper, we introduce ScenePilot-4K, a large-scale first-person dataset for safety-aware vision-language learning and evaluation in autonomous driving. Built from public online driving videos, ScenePilot-4K contains 3,847 hours of video and 27.7M front-view frames spanning 63 countries/regions and 1,210 cities. It jointly provides scene-level natural-language descriptions, risk assessment labels, key-participant annotations, ego trajectories, and camera parameters through a unified multi-stage annotation pipeline. Building on this dataset, we establish ScenePilot-Bench, a standardized benchmark that evaluates vision-language models along four complementary axes: scene understanding, spatial perception, motion planning, and GPT-based semantic alignment. The benchmark includes fine-grained metrics and geographic generalization settings that expose model robustness under cross-region and cross-traffic domain shifts. Baseline results on representative open-source and proprietary vision-language models show that current models remain competitive in high-level scene semantics but still exhibit substantial limitations in geometry-aware perception and planning-oriented reasoning. Beyond the released dataset itself, the proposed annotation pipeline serves as a reusable and extensible recipe for scalable dataset construction from public Internet driving videos.
Zitieren
APA
Wang, Y., Zheng, Y., Fan, W., Wang, T., Chu, H., Zhang, L., Gao, B., Tian, D., Wang, J., & Chen, H. (2026). ScenePilot-4K: A Large-Scale First-Person Dataset and Benchmark for Vision-Language Models in Autonomous Driving. Communications in Transportation Research. https://doi.org/10.26599/commtr.2026.9640069
Vancouver
Wang Y, Zheng Y, Fan W, Wang T, Chu H, Zhang L, et al. ScenePilot-4K: A Large-Scale First-Person Dataset and Benchmark for Vision-Language Models in Autonomous Driving. Communications in Transportation Research. 2026. doi:10.26599/commtr.2026.9640069
BibTeX
@article{Wang2026,
author = {Wang, Yujin and Zheng, Yutong and Fan, Wenxian and Wang, Tianyi and Chu, Hongqing and Zhang, Li and Gao, Bingzhao and Tian, Daxin and Wang, Jianqiang and Chen, Hong},
title = {ScenePilot-4K: A Large-Scale First-Person Dataset and Benchmark for Vision-Language Models in Autonomous Driving},
journal = {Communications in Transportation Research},
year = {2026},
doi = {10.26599/commtr.2026.9640069}
}
EUVIMED