EUVIMEDEUVIMED entdecken
Crossref

A Multi-Modal Architecture With Spatio-Temporal-Text Adaptation for Video-Based Traffic Accident Anticipation

Patrik Patera, Yie-Tarng Chen, Wen-Hsien Fang

IEEE Transactions on Circuits and Systems for Video Technology · 2025-09 · 35 (9) · 8989-9002

DOI: 10.1109/tcsvt.2025.3552895 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-09. Verfasst von Patrik Patera und 2 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 56 Literaturangaben. Gefördert von: National Science and Technology Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Early and precise accident anticipation is critical for preventing road traffic incidents in advanced traffic systems. This paper presents a Multi-modal Architecture with Spatio-Temporal-Text Adaptation (MASTTA), featuring a Visual Encoder and a Text Encoder within a streamlined end-to-end framework for traffic accident anticipation. Both encoders leverage the CLIP model, pre-trained on large-scale text-image pairs, to utilize visual and textual information effectively. MASTTA captures complex traffic patterns and relationships by fine-tuning only the adapters, reducing retraining demands. In the Visual Encoder, spatio-temporal adaptation is achieved through a novel Temporal Adapter, a novel Spatial Adapter, and an MLP Adapter. The Temporal Adapter enhances temporal consistency in accident-prone areas, while the Spatial Adapter captures spatio-temporal interactions among visual cues. The Text Encoder, equipped with a Text Adapter and an MLP Adapter, aligns latent textual and visual features in a joint embedding space, refining semantic representation. This synergy of text and visual adapters enables MASTTA to model complex spatial interactions across long-range temporal context, improving accident anticipation. We validate MASTTA on DAD and CCD datasets, demonstrating significant improvements in both the earliness and correctness compared to state-of-the-art methods.

Förderung

  • National Science and Technology Council

Zitieren

APA

Patera, P., Chen, Y., & Fang, W. (2025). A Multi-Modal Architecture With Spatio-Temporal-Text Adaptation for Video-Based Traffic Accident Anticipation. IEEE Transactions on Circuits and Systems for Video Technology, 35(9), 8989-9002. https://doi.org/10.1109/tcsvt.2025.3552895

Vancouver

Patera P, Chen Y, Fang W. A Multi-Modal Architecture With Spatio-Temporal-Text Adaptation for Video-Based Traffic Accident Anticipation. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(9):8989-9002. doi:10.1109/tcsvt.2025.3552895

BibTeX
@article{Patera2025,
  author = {Patera, Patrik and Chen, Yie-Tarng and Fang, Wen-Hsien},
  title = {A Multi-Modal Architecture With Spatio-Temporal-Text Adaptation for Video-Based Traffic Accident Anticipation},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {9},
  pages = {8989--9002},
  doi = {10.1109/tcsvt.2025.3552895}
}
Kostenlos starten →