EUVIMEDEUVIMED entdecken
Crossref

Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models

Yifang Xu, Yunzhuo Sun, Benxiang Zhai, Ming Li, Wenxin Liang, Yang Li, Sidan Du

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (9) · 8978-8986

DOI: 10.1609/aaai.v39i9.32971 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Yifang Xu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The target of video moment retrieval (VMR) is predicting temporal spans within a video that semantically match a given linguistic query. Existing VMR methods based on multimodal large language models (MLLMs) overly rely on expensive high-quality datasets and time-consuming fine-tuning. Although some recent studies introduce a zero-shot setting to avoid fine-tuning, they overlook inherent language bias in the query, leading to erroneous localization. To tackle the aforementioned challenges, this paper proposes Moment-GPT, a tuning-free pipeline for zero-shot VMR utilizing frozen MLLMs. Specifically, we first employ LLaMA-3 to correct and rephrase the query to mitigate language bias. Subsequently, we design a span generator combined with MiniGPT-v2 to produce candidate spans adaptively. Finally, to leverage the video comprehension capabilities of MLLMs, we apply Video-ChatGPT and span scorer to select the most appropriate spans. Our proposed method substantially outperforms the state-of-the-art MLLM-based and zero-shot models on several public datasets, including QVHighlights, ActivityNet-Captions, and Charades-STA.

Zitieren

APA

Xu, Y., Sun, Y., Zhai, B., Li, M., Liang, W., Li, Y., & Du, S. (2025). Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 39(9), 8978-8986. https://doi.org/10.1609/aaai.v39i9.32971

Vancouver

Xu Y, Sun Y, Zhai B, Li M, Liang W, Li Y, et al. Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(9):8978-8986. doi:10.1609/aaai.v39i9.32971

BibTeX
@article{Xu2025,
  author = {Xu, Yifang and Sun, Yunzhuo and Zhai, Benxiang and Li, Ming and Liang, Wenxin and Li, Yang and Du, Sidan},
  title = {Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {9},
  pages = {8978--8986},
  doi = {10.1609/aaai.v39i9.32971}
}
Kostenlos starten →