EUVIMEDEUVIMED entdecken
Crossref

Learning Dynamic Similarity by Bidirectional Hierarchical Sliding Semantic Probe for Efficient Text Video Retrieval

Yang Liu, Shudong Huang, Deng Xiong, Jiancheng Lv

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (6) · 5667-5675

DOI: 10.1609/aaai.v39i6.32604 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Yang Liu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Text-video retrieval is a foundation task in multi-modal research which aims to align texts and videos in the embedding space. The key challenge is to learn the similarity between videos and texts. A conventional approach involves directly aligning video-text pairs using cosine similarity. However, due to the disparity in the information conveyed by videos and texts, i.e., a single video can be described from multiple perspectives, the retrieval accuracy is suboptimal. An alternative approach employs cross-modal interaction to enable videos to dynamically acquire distinct features from various texts, thus facilitating similarity calculations. Nevertheless, this solution incurs a computational complexity of O(n^2) during retrieval. To this end, this paper proposes a novel method called Bidirectional Hierarchical Sliding Semantic Probe (BiHSSP), which calculates dynamic similarity between videos and texts with O(n) complexity during retrieval. We introduce a hierarchical semantic probe module that learns semantic probes at different scales for both video and text features. Semantic probe involves a sliding calculation of the cross-correlation between semantic probes at different scales and embeddings from another modality, allowing for dynamic similarity computation between video and text descriptions from various perspectives. Specifically, for text descriptions from different angles, we calculate the similarity at different locations within the video features and vice versa. This approach preserves the complete information of the video while addressing the issue of unequal information between video and text without requiring cross-modal interaction. Additionally, our method can function as a plug-and-play module across various methods, thereby enhancing the corresponding performance. Experimental results demonstrate that our BiHSSP significantly outperforms the baseline.

Zitieren

APA

Liu, Y., Huang, S., Xiong, D., & Lv, J. (2025). Learning Dynamic Similarity by Bidirectional Hierarchical Sliding Semantic Probe for Efficient Text Video Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, 39(6), 5667-5675. https://doi.org/10.1609/aaai.v39i6.32604

Vancouver

Liu Y, Huang S, Xiong D, Lv J. Learning Dynamic Similarity by Bidirectional Hierarchical Sliding Semantic Probe for Efficient Text Video Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(6):5667-5675. doi:10.1609/aaai.v39i6.32604

BibTeX
@article{Liu2025,
  author = {Liu, Yang and Huang, Shudong and Xiong, Deng and Lv, Jiancheng},
  title = {Learning Dynamic Similarity by Bidirectional Hierarchical Sliding Semantic Probe for Efficient Text Video Retrieval},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {6},
  pages = {5667--5675},
  doi = {10.1609/aaai.v39i6.32604}
}
Kostenlos starten →