Learning a Conceptual Space as a Bridge for Text-Video Retrieval
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026 · 1-15
DOI: 10.1109/tpami.2026.3751274 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026. Verfasst von Yizhe Li und 4 weiteren Autor:innen.
Abstract (Originalsprache)
The goal of text-video retrieval is to learn a shared embedding space in which semantically relevant videos and texts have higher similarity. Recent advances employ contrastive learning to bring the paired video and text features closer. Despite these achievements, the inherent modality disparity, i.e., videos are detailed while texts are often brief, leads to the issues of incomplete correspondence and false negatives between video and text. These issues limit the effectiveness of existing contrastive learning approaches that rely on hard labels for direct text-video alignment. To this end, we present CoSTVR, a novel framework that introduces a conceptual space as a semantic bridge to enhance text-video feature alignment. Specifically, we regard concepts as a shared representation that bridges video and text modalities, and construct a conceptual codebook based on a set of concept prototypes. Building upon this codebook, we perform complementary concept-centered alignment in addition to direct text-video alignment. To implement concept-centered alignment from different feature granularities, we design token quantization and concept perception modules. Token quantization maps continuous patch-level and word-level features to discrete conceptual embeddings. Concept perception predicts the concepts present in video and text from global features, supervised by the concept labels we constructed. These two modules guide video and text features toward shared conceptual representations, grounding both modalities in a unified conceptual space. Extensive experiments demonstrate the effectiveness of CoSTVR, achieving state of-the-art performance on three popular benchmarks, MSRVTT, DiDeMo, and ActivityNet Captions. Further ablation analysis shows that our method effectively mitigates the modality gap and enhances text-video alignment.
Zitieren
APA
Li, Y., Zhou, S., Huo, H., Dong, Y., & Wang, L. (2026). Learning a Conceptual Space as a Bridge for Text-Video Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1-15. https://doi.org/10.1109/tpami.2026.3751274
Vancouver
Li Y, Zhou S, Huo H, Dong Y, Wang L. Learning a Conceptual Space as a Bridge for Text-Video Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026:1-15. doi:10.1109/tpami.2026.3751274
BibTeX
@article{Li2026,
author = {Li, Yizhe and Zhou, Sanping and Huo, Hongcheng and Dong, Yonghao and Wang, Le},
title = {Learning a Conceptual Space as a Bridge for Text-Video Retrieval},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
pages = {1--15},
doi = {10.1109/tpami.2026.3751274}
}
EUVIMED