Spatio-Temporal Data Enhanced Vision-Language Model for Traffic Scene Understanding
IEEE Transactions on Intelligent Transportation Systems · 2026-01 · 27 (1) · 1248-1266
DOI: 10.1109/tits.2025.3628271 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Intelligent Transportation Systems, 2026-01. Verfasst von Jingtian Ma und 4 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 70 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, DiDi Gaia Collaborative Research Funds.
Abstract (Originalsprache)
Nowadays, navigation and ride-sharing apps have collected numerous images with spatio-temporal data. A core technology for analyzing such images, associated with spatio-temporal information, is Traffic Scene Understanding (TSU), which aims to provide a comprehensive description of the traffic scene. Unlike traditional spatio-temporal data analysis tasks, the dependence on both spatio-temporal and visual-textual data introduces distinct challenges to TSU task. However, recent research often treats TSU as a common image understanding task, ignoring the spatio-temporal information and overlooking the interrelations between different aspects of the traffic scene. To address these issues, we propose a novel Spatio-Temporal Enhanced Model based on CILP (ST-CLIP) for TSU. Our model uses the classic vision-language model, CLIP, as the backbone, and designs a Spatio-temporal Context Aware Multi-aspect Prompt (SCAMP) learning method to incorporate spatio-temporal information into TSU. The prompt learning method consists of two components: A dynamic spatio-temporal context representation module that extracts representation vectors of spatio-temporal data for each traffic scene image, and a bi-level ST-aware multi-aspect prompt learning module that integrates the ST-context representation vectors into word embeddings of prompts for the CLIP model. The second module also extracts low-level visual features and image-wise high-level semantic features to exploit interactive relations among different aspects of traffic scenes. To the best of our knowledge, this is the first attempt to integrate spatio-temporal information into vision-language models to facilitate TSU task. Experiments on two real-world datasets demonstrate superior performance in the complex scene understanding scenarios with a few-shot learning strategy.
Förderung
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities
- DiDi Gaia Collaborative Research Funds
Zitieren
APA
Ma, J., Wang, J., Zhao, W. X., Liu, G., & Wen, X. (2026). Spatio-Temporal Data Enhanced Vision-Language Model for Traffic Scene Understanding. IEEE Transactions on Intelligent Transportation Systems, 27(1), 1248-1266. https://doi.org/10.1109/tits.2025.3628271
Vancouver
Ma J, Wang J, Zhao WX, Liu G, Wen X. Spatio-Temporal Data Enhanced Vision-Language Model for Traffic Scene Understanding. IEEE Transactions on Intelligent Transportation Systems. 2026;27(1):1248-1266. doi:10.1109/tits.2025.3628271
BibTeX
@article{Ma2026,
author = {Ma, Jingtian and Wang, Jingyuan and Zhao, Wayne Xin and Liu, Guoping and Wen, Xiang},
title = {Spatio-Temporal Data Enhanced Vision-Language Model for Traffic Scene Understanding},
journal = {IEEE Transactions on Intelligent Transportation Systems},
year = {2026},
volume = {27},
number = {1},
pages = {1248--1266},
doi = {10.1109/tits.2025.3628271}
}
EUVIMED