Bridging the gap: multi-granularity representation learning for text-based vehicle retrieval
Complex & Intelligent Systems · 2025-01 · 11 (1)
DOI: 10.1007/s40747-024-01614-w ↗Kurz gefasst
Fachartikel in der Zeitschrift Complex & Intelligent Systems, online erschienen am 13.11.2024. Verfasst von Xue Bo und 3 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Innovative Research Group Project of the National Natural Science Foundation of China.
Abstract (Originalsprache)
Text-based cross-modal vehicle retrieval has been widely applied in smart city contexts and other scenarios. The objective of this approach is to identify semantically relevant target vehicles in videos using text descriptions, thereby facilitating the analysis of vehicle spatio-temporal trajectories. Current methodologies predominantly employ a two-tower architecture, where single-granularity features from both visual and textual domains are extracted independently. However, due to the intricate semantic relationships between videos and text, aligning the two modalities effectively using single-granularity feature representation poses a challenge. To address this issue, we introduce a Multi-Granularity Representation Learning model, termed MGRL, tailored for text-based cross-modal vehicle retrieval. Specifically, the model parses information from the two modalities into three hierarchical levels of feature representation: coarse-granularity, medium-granularity, and fine-granularity. Subsequently, a feature adaptive fusion strategy is devised to automatically determine the optimal pooling mechanism. Finally, a multi-granularity contrastive learning approach is implemented to ensure comprehensive semantic coverage, ranging from coarse to fine levels. Experimental outcomes on public benchmarks show that our method achieves up to a 14.56% improvement in text-to-vehicle retrieval performance, as measured by the Mean Reciprocal Rank (MRR) metric, when compared against 10 state-of-the-art baselines and 6 ablation studies.
Förderung
- Innovative Research Group Project of the National Natural Science Foundation of China
Zitieren
APA
Bo, X., Liu, J., Yang, D., & Ma, W. (2025). Bridging the gap: multi-granularity representation learning for text-based vehicle retrieval. Complex & Intelligent Systems, 11(1). https://doi.org/10.1007/s40747-024-01614-w
Vancouver
Bo X, Liu J, Yang D, Ma W. Bridging the gap: multi-granularity representation learning for text-based vehicle retrieval. Complex & Intelligent Systems. 2025;11(1). doi:10.1007/s40747-024-01614-w
BibTeX
@article{Bo2025,
author = {Bo, Xue and Liu, Junjie and Yang, Di and Ma, Wentao},
title = {Bridging the gap: multi-granularity representation learning for text-based vehicle retrieval},
journal = {Complex & Intelligent Systems},
year = {2025},
volume = {11},
number = {1},
doi = {10.1007/s40747-024-01614-w}
}
EUVIMED