EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

SSTIR: Harnessing Multimodal Large-Language Models for Semi-Structured Traffic Incident Reports

Mirabela-Melinda Medvei, George-Andrei Dima, Stefania Stefanescu

IEEE Access · 2026 · 14 · 164271-164287

DOI: 10.1109/access.2026.3744224 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Mirabela-Melinda Medvei und 2 weiteren Autor:innen. Die Arbeit nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National University of Science and Technology Politehnica Bucharest, through the institutional Open access program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present SSTIR, the Semi-Structured Traffic Incident Reports corpus, a novel, publicly available multimodal dataset designed to advance the automated reporting and analysis of traffic incidents. SSTIR is the first dataset to capture video snippets of real-world traffic incidents, together with human-annotated reports that are partly structured and partly free text, contributing to both Urban Foundation Models (UFM) and Intelligent Transportation Systems (ITS). Each annotation includes incident details such as location, context, severity, contributing factors, environmental conditions, immediate outcomes, and actors involved. We benchmark four representative open-domain vision-language models, LLaVA-1.6, Qwen2-VL-7B, Gemma-3-4B, and Llama 3.2-11B, in a zero-shot setting on SSTIR’s schema-driven reporting task, finding that all four exhibit only moderate semantic alignment, weak factual correctness, and limited actor-level grounding, with Llama 3.2-11B emerging as the strongest overall baseline. Motivated by this finding, we fine-tune Llama-SSTIR from Llama 3.2 Vision using parameter-efficient methods. We also investigate semantic-based evaluation metrics, identifying BERTScore and the “LLM-as-a-judge” approach as complementary to traditional lexical metrics such as BLEU and ROUGE for assessing the quality of semi-structured reports. The evaluation demonstrates that Llama-SSTIR substantially outperforms all four zero-shot baselines, as well as one-shot prompting, with particularly strong gains in semantic alignment, output validity, and actor-type detection, confirming the value of schema-aware fine-tuning.

Förderung

  • National University of Science and Technology Politehnica Bucharest, through the institutional Open access program

Zitieren

APA

Medvei, M., Dima, G., & Stefanescu, S. (2026). SSTIR: Harnessing Multimodal Large-Language Models for Semi-Structured Traffic Incident Reports. IEEE Access, 14, 164271-164287. https://doi.org/10.1109/access.2026.3744224

Vancouver

Medvei M, Dima G, Stefanescu S. SSTIR: Harnessing Multimodal Large-Language Models for Semi-Structured Traffic Incident Reports. IEEE Access. 2026;14:164271-164287. doi:10.1109/access.2026.3744224

BibTeX
@article{Medvei2026,
  author = {Medvei, Mirabela-Melinda and Dima, George-Andrei and Stefanescu, Stefania},
  title = {SSTIR: Harnessing Multimodal Large-Language Models for Semi-Structured Traffic Incident Reports},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {164271--164287},
  doi = {10.1109/access.2026.3744224}
}
Kostenlos starten →