EUVIMEDEUVIMED entdecken
Crossref

Spatio-Temporal Data Enhanced Vision-Language Model for Traffic Scene Understanding

Jingtian Ma, Jingyuan Wang, Wayne Xin Zhao, Guoping Liu, Xiang Wen

IEEE Transactions on Intelligent Transportation Systems · 2026-01 · 27 (1) · 1248-1266

DOI: 10.1109/tits.2025.3628271 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Intelligent Transportation Systems, 2026-01. Verfasst von Jingtian Ma und 4 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 70 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, DiDi Gaia Collaborative Research Funds.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Nowadays, navigation and ride-sharing apps have collected numerous images with spatio-temporal data. A core technology for analyzing such images, associated with spatio-temporal information, is Traffic Scene Understanding (TSU), which aims to provide a comprehensive description of the traffic scene. Unlike traditional spatio-temporal data analysis tasks, the dependence on both spatio-temporal and visual-textual data introduces distinct challenges to TSU task. However, recent research often treats TSU as a common image understanding task, ignoring the spatio-temporal information and overlooking the interrelations between different aspects of the traffic scene. To address these issues, we propose a novel Spatio-Temporal Enhanced Model based on CILP (ST-CLIP) for TSU. Our model uses the classic vision-language model, CLIP, as the backbone, and designs a Spatio-temporal Context Aware Multi-aspect Prompt (SCAMP) learning method to incorporate spatio-temporal information into TSU. The prompt learning method consists of two components: A dynamic spatio-temporal context representation module that extracts representation vectors of spatio-temporal data for each traffic scene image, and a bi-level ST-aware multi-aspect prompt learning module that integrates the ST-context representation vectors into word embeddings of prompts for the CLIP model. The second module also extracts low-level visual features and image-wise high-level semantic features to exploit interactive relations among different aspects of traffic scenes. To the best of our knowledge, this is the first attempt to integrate spatio-temporal information into vision-language models to facilitate TSU task. Experiments on two real-world datasets demonstrate superior performance in the complex scene understanding scenarios with a few-shot learning strategy.

Förderung

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities
  • DiDi Gaia Collaborative Research Funds

Zitieren

APA

Ma, J., Wang, J., Zhao, W. X., Liu, G., & Wen, X. (2026). Spatio-Temporal Data Enhanced Vision-Language Model for Traffic Scene Understanding. IEEE Transactions on Intelligent Transportation Systems, 27(1), 1248-1266. https://doi.org/10.1109/tits.2025.3628271

Vancouver

Ma J, Wang J, Zhao WX, Liu G, Wen X. Spatio-Temporal Data Enhanced Vision-Language Model for Traffic Scene Understanding. IEEE Transactions on Intelligent Transportation Systems. 2026;27(1):1248-1266. doi:10.1109/tits.2025.3628271

BibTeX
@article{Ma2026,
  author = {Ma, Jingtian and Wang, Jingyuan and Zhao, Wayne Xin and Liu, Guoping and Wen, Xiang},
  title = {Spatio-Temporal Data Enhanced Vision-Language Model for Traffic Scene Understanding},
  journal = {IEEE Transactions on Intelligent Transportation Systems},
  year = {2026},
  volume = {27},
  number = {1},
  pages = {1248--1266},
  doi = {10.1109/tits.2025.3628271}
}
Kostenlos starten →