Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (9) · 8978-8986
DOI: 10.1609/aaai.v39i9.32971 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Yifang Xu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref).
Abstract (Originalsprache)
The target of video moment retrieval (VMR) is predicting temporal spans within a video that semantically match a given linguistic query. Existing VMR methods based on multimodal large language models (MLLMs) overly rely on expensive high-quality datasets and time-consuming fine-tuning. Although some recent studies introduce a zero-shot setting to avoid fine-tuning, they overlook inherent language bias in the query, leading to erroneous localization. To tackle the aforementioned challenges, this paper proposes Moment-GPT, a tuning-free pipeline for zero-shot VMR utilizing frozen MLLMs. Specifically, we first employ LLaMA-3 to correct and rephrase the query to mitigate language bias. Subsequently, we design a span generator combined with MiniGPT-v2 to produce candidate spans adaptively. Finally, to leverage the video comprehension capabilities of MLLMs, we apply Video-ChatGPT and span scorer to select the most appropriate spans. Our proposed method substantially outperforms the state-of-the-art MLLM-based and zero-shot models on several public datasets, including QVHighlights, ActivityNet-Captions, and Charades-STA.
Zitieren
APA
Xu, Y., Sun, Y., Zhai, B., Li, M., Liang, W., Li, Y., & Du, S. (2025). Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 39(9), 8978-8986. https://doi.org/10.1609/aaai.v39i9.32971
Vancouver
Xu Y, Sun Y, Zhai B, Li M, Liang W, Li Y, et al. Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(9):8978-8986. doi:10.1609/aaai.v39i9.32971
BibTeX
@article{Xu2025,
author = {Xu, Yifang and Sun, Yunzhuo and Zhai, Benxiang and Li, Ming and Liang, Wenxin and Li, Yang and Du, Sidan},
title = {Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {9},
pages = {8978--8986},
doi = {10.1609/aaai.v39i9.32971}
}
EUVIMED