EUVIMEDEUVIMED entdecken
Crossref

Bidirectional Cross-Modal Collaborative Alignment via Semantic-Guided Visual Embeddings for Partially Relevant Video Retrieval

Huafeng Li, Jialong Zhao, Yafei Zhang, Jie Wen

IEEE Transactions on Image Processing · 2026 · 35 · 1423-1435

DOI: 10.1109/tip.2026.3658218 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2026. Verfasst von Huafeng Li und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 51 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Yunnan Fundamental Research Projects.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Partially Relevant Video Retrieval (PRVR) aims to retrieve videos that match a given textual query only partially. This task is inherently challenging due to the modality gap between text and video, which is further exacerbated by the partial semantic correspondence between linguistic descriptions and visual content. To address these challenges, we propose a bidirectional cross-modal alignment mechanism that collaboratively optimizes both visual and textual modalities. In the visual modality, a major difficulty lies in the absence of visual cues that directly correspond to textual semantics, limiting the model's ability to align visual representations with textual meanings under unsupervised conditions. To overcome this issue, we construct a semantic-visual association library, which stores paired visual and textual features with semantic annotations. During training, the model dynamically retrieves the most semantically similar visual samples from this library based on the current visual feature vector. These retrieved samples, preliminarily associated with semantics via cross-modal matching, are used to form dynamic anchors that guide visual representation learning. By leveraging these enriched visual features, the model progressively refines the visual representations to achieve better alignment with the corresponding textual inputs, thereby enhancing cross-modal consistency. In the textual modality, we enhance textual representations by integrating semantically aligned visual features selected from the same association library, further narrowing the modality gap. Extensive experiments on benchmark datasets under partial semantic correspondence scenarios demonstrate that our method achieves state-of-the-art performance. The source code of the paper is available at https://github.com/cyanlll/BOA.

Förderung

  • National Natural Science Foundation of China
  • Yunnan Fundamental Research Projects

Zitieren

APA

Li, H., Zhao, J., Zhang, Y., & Wen, J. (2026). Bidirectional Cross-Modal Collaborative Alignment via Semantic-Guided Visual Embeddings for Partially Relevant Video Retrieval. IEEE Transactions on Image Processing, 35, 1423-1435. https://doi.org/10.1109/tip.2026.3658218

Vancouver

Li H, Zhao J, Zhang Y, Wen J. Bidirectional Cross-Modal Collaborative Alignment via Semantic-Guided Visual Embeddings for Partially Relevant Video Retrieval. IEEE Transactions on Image Processing. 2026;35:1423-1435. doi:10.1109/tip.2026.3658218

BibTeX
@article{Li2026,
  author = {Li, Huafeng and Zhao, Jialong and Zhang, Yafei and Wen, Jie},
  title = {Bidirectional Cross-Modal Collaborative Alignment via Semantic-Guided Visual Embeddings for Partially Relevant Video Retrieval},
  journal = {IEEE Transactions on Image Processing},
  year = {2026},
  volume = {35},
  pages = {1423--1435},
  doi = {10.1109/tip.2026.3658218}
}
Kostenlos starten →