Vision Language Model–Guided Semantic Distillation Network for Pedestrian Crossing Intention Prediction
Communications in Transportation Research · 26.10.2026
DOI: 10.26599/commtr.2026.9640061 ↗Kurz gefasst
Fachartikel in der Zeitschrift Communications in Transportation Research, online erschienen am 26.10.2026. Verfasst von Yuanzhe Li und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Pedestrian crossing intention prediction is a critical function of autonomous vehicles, which enables safe pedestrian–vehicle interactions in complex urban environments. However, accurately predicting pedestrian intention remains challenging, as it depends on multiple factors, including pedestrian behavior, vehicle dynamics, and surrounding context. To improve prediction performance, existing approaches increasingly incorporate more modalities, but this often results in substantial computational overhead. Achieving accurate intention prediction with a limited number of modalities remains an open challenge. In this paper, we propose a vision language model-guided semantic distillation network (VLM-GSDN) for pedestrian crossing intention prediction. During training, VLM-GSDN leverages pedestrian- and scene-level semantic descriptions generated by a vision language model to align local and global visual representations with their corresponding high-level semantics, while requiring only RGB images and image-space pedestrian motion for efficient inference. To incorporate motion cues, we introduce motion-guided modulation and predictive spatial modulation, which refine pedestrian representations using pedestrian motion and predicted future displacements. A selective temporal attention module is further incorporated to integrate temporal information and emphasize informative frames. Moreover, a multi-task learning strategy is adopted in which pedestrian action-state transition prediction serves as an auxiliary task to provide additional motion supervision. Extensive experiments on the JAAD and PIE datasets demonstrate that VLM-GSDN achieves competitive performance compared with state-of-the-art methods. Moreover, it exhibits promising cross-dataset generalization and maintains robust performance over longer prediction horizons.
Zitieren
APA
Li, Y., Wang, C., Zeng, Z., & Müller, S. (2026). Vision Language Model–Guided Semantic Distillation Network for Pedestrian Crossing Intention Prediction. Communications in Transportation Research. https://doi.org/10.26599/commtr.2026.9640061
Vancouver
Li Y, Wang C, Zeng Z, Müller S. Vision Language Model–Guided Semantic Distillation Network for Pedestrian Crossing Intention Prediction. Communications in Transportation Research. 2026. doi:10.26599/commtr.2026.9640061
BibTeX
@article{Li2026,
author = {Li, Yuanzhe and Wang, Cheng and Zeng, Zhuoqi and Müller, Steffen},
title = {Vision Language Model–Guided Semantic Distillation Network for Pedestrian Crossing Intention Prediction},
journal = {Communications in Transportation Research},
year = {2026},
doi = {10.26599/commtr.2026.9640061}
}
EUVIMED