EUVIMEDEUVIMED entdecken
Crossref

V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning

Hang Hua, Yunlong Tang, Chenliang Xu, Jiebo Luo

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (4) · 3599-3607

DOI: 10.1609/aaai.v39i4.32374 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Hang Hua und 3 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video summarization aims to create short, accurate, and cohesive summaries of longer videos. Despite the existence of various video summarization datasets, a notable limitation is their limited amount of source videos, which hampers the effective training of advanced large vision-language models (VLMs). Additionally, most existing datasets are created for video-to-video summarization, overlooking the contemporary need for multimodal video content summarization. Recent efforts have been made to expand from unimodal to multimodal video summarization, categorizing the task into three sub-tasks based on the summary's modality: video-to-video (V2V), video-to-text (V2T), and a combination of video and text summarization (V2VT). However, the textual summaries in previous multimodal datasets are inadequate. To address these issues, we introduce Instruct-V2Xum, a cross-modal video summarization dataset featuring 30,000 diverse videos sourced from YouTube, with lengths ranging from 40 to 940 seconds and an average summarization ratio of 16.39%. Each video summary in Instruct-V2Xum is paired with a textual summary that references specific frame indexes, facilitating the generation of aligned video and textual summaries. In addition, we propose a new video summarization framework named V2Xum-LLM. V2Xum-LLM, specifically V2Xum-LLaMA in this study, is the first framework that unifies different video summarization tasks into one large language model's (LLM) text decoder and achieves task-controllable video summarization with temporal prompts and task instructions. Experiments show that V2Xum-LLaMA outperforms strong baseline models on multiple video summarization tasks. Furthermore, we propose an enhanced evaluation metric for V2V and V2VT summarization tasks.

Zitieren

APA

Hua, H., Tang, Y., Xu, C., & Luo, J. (2025). V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning. Proceedings of the AAAI Conference on Artificial Intelligence, 39(4), 3599-3607. https://doi.org/10.1609/aaai.v39i4.32374

Vancouver

Hua H, Tang Y, Xu C, Luo J. V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(4):3599-3607. doi:10.1609/aaai.v39i4.32374

BibTeX
@article{Hua2025,
  author = {Hua, Hang and Tang, Yunlong and Xu, Chenliang and Luo, Jiebo},
  title = {V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {4},
  pages = {3599--3607},
  doi = {10.1609/aaai.v39i4.32374}
}
Kostenlos starten →