Bidirectional Cross-Modal Collaborative Alignment via Semantic-Guided Visual Embeddings for Partially Relevant Video Retrieval
IEEE Transactions on Image Processing · 2026 · 35 · 1423-1435
DOI: 10.1109/tip.2026.3658218 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2026. Verfasst von Huafeng Li und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 51 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Yunnan Fundamental Research Projects.
Abstract (Originalsprache)
Partially Relevant Video Retrieval (PRVR) aims to retrieve videos that match a given textual query only partially. This task is inherently challenging due to the modality gap between text and video, which is further exacerbated by the partial semantic correspondence between linguistic descriptions and visual content. To address these challenges, we propose a bidirectional cross-modal alignment mechanism that collaboratively optimizes both visual and textual modalities. In the visual modality, a major difficulty lies in the absence of visual cues that directly correspond to textual semantics, limiting the model's ability to align visual representations with textual meanings under unsupervised conditions. To overcome this issue, we construct a semantic-visual association library, which stores paired visual and textual features with semantic annotations. During training, the model dynamically retrieves the most semantically similar visual samples from this library based on the current visual feature vector. These retrieved samples, preliminarily associated with semantics via cross-modal matching, are used to form dynamic anchors that guide visual representation learning. By leveraging these enriched visual features, the model progressively refines the visual representations to achieve better alignment with the corresponding textual inputs, thereby enhancing cross-modal consistency. In the textual modality, we enhance textual representations by integrating semantically aligned visual features selected from the same association library, further narrowing the modality gap. Extensive experiments on benchmark datasets under partial semantic correspondence scenarios demonstrate that our method achieves state-of-the-art performance. The source code of the paper is available at https://github.com/cyanlll/BOA.
Förderung
- National Natural Science Foundation of China
- Yunnan Fundamental Research Projects
Zitieren
APA
Li, H., Zhao, J., Zhang, Y., & Wen, J. (2026). Bidirectional Cross-Modal Collaborative Alignment via Semantic-Guided Visual Embeddings for Partially Relevant Video Retrieval. IEEE Transactions on Image Processing, 35, 1423-1435. https://doi.org/10.1109/tip.2026.3658218
Vancouver
Li H, Zhao J, Zhang Y, Wen J. Bidirectional Cross-Modal Collaborative Alignment via Semantic-Guided Visual Embeddings for Partially Relevant Video Retrieval. IEEE Transactions on Image Processing. 2026;35:1423-1435. doi:10.1109/tip.2026.3658218
BibTeX
@article{Li2026,
author = {Li, Huafeng and Zhao, Jialong and Zhang, Yafei and Wen, Jie},
title = {Bidirectional Cross-Modal Collaborative Alignment via Semantic-Guided Visual Embeddings for Partially Relevant Video Retrieval},
journal = {IEEE Transactions on Image Processing},
year = {2026},
volume = {35},
pages = {1423--1435},
doi = {10.1109/tip.2026.3658218}
}
EUVIMED