EUVIMEDEUVIMED entdecken
Crossref

Transferable Video Moment Localization by Moment-Guided Query Prompting

Hao Jiang, Yang Yizhang, Yadong Mu

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (3) · 2516-2524

DOI: 10.1609/aaai.v38i3.28028 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Hao Jiang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video moment localization stands as a crucial task within the realm of computer vision, entailing the identification of temporal moments in untrimmed videos that bear semantic relevance to the supplied natural language queries. This work delves into a relatively unexplored facet of the task: the transferability of video moment localization models. This concern is addressed by evaluating moment localization models within a cross-domain transfer setting. In this setup, we curate multiple datasets distinguished by substantial domain gaps. The model undergoes training on one of these datasets, while validation and testing are executed using the remaining datasets. To confront the challenges inherent in this scenario, we draw inspiration from the recently introduced large-scale pre-trained vision-language models. Our focus is on exploring how the strategic utilization of these resources can bolster the capabilities of a model designed for video moment localization. Nevertheless, the distribution of language queries in video moment localization usually diverges from the text used by pre-trained models, exhibiting distinctions in aspects such as length, content, expression, and more. To mitigate the gap, this work proposes a Moment-Guided Query Prompting (MGQP) method for video moment localization. Our key idea is to generate multiple distinct and complementary prompt primitives through stratification of the original queries. Our approach is comprised of a prompt primitive constructor, a multimodal prompt refiner, and a holistic prompt incorporator. We carry out extensive experiments on Charades-STA, TACoS, DiDeMo, and YouCookII datasets, and investigate the efficacy of the proposed method using various pre-trained models, such as CLIP, ActionCLIP, CLIP4Clip, and VideoCLIP. The experimental results demonstrate the effectiveness of our proposed method.

Zitieren

APA

Jiang, H., Yizhang, Y., & Mu, Y. (2024). Transferable Video Moment Localization by Moment-Guided Query Prompting. Proceedings of the AAAI Conference on Artificial Intelligence, 38(3), 2516-2524. https://doi.org/10.1609/aaai.v38i3.28028

Vancouver

Jiang H, Yizhang Y, Mu Y. Transferable Video Moment Localization by Moment-Guided Query Prompting. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(3):2516-2524. doi:10.1609/aaai.v38i3.28028

BibTeX
@article{Jiang2024,
  author = {Jiang, Hao and Yizhang, Yang and Mu, Yadong},
  title = {Transferable Video Moment Localization by Moment-Guided Query Prompting},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {3},
  pages = {2516--2524},
  doi = {10.1609/aaai.v38i3.28028}
}
Kostenlos starten →