EUVIMEDEUVIMED entdecken
Crossref

VMChill: A Dataset for Fine-Grained Visual-Musical Synergy

Xiaowei Chi, Zeyue Tian, Jialiang Chen, Wei Xue

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (5) · 3353-3362

DOI: 10.1609/aaai.v40i5.37331 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xiaowei Chi und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Massive multi-modality datasets are fundamental to the success of large video-language models. However, existing datasets often focus on providing textual descriptions for visual content, treating audio, particularly music, as weakly related information. This overlooks the inherent semantic correlation between visual narratives and musical scores, limiting the development of models for fine-grained cross-modal understanding and generation. To address this gap, we introduce VMChill, a large-scale, fine-grained multimodal video dataset. We leverage trailers as our data source, as they are professionally edited to create a strong synergy between visual pacing, scene transitions, and background music for narrative and emotional impact. Our dataset comprises over 20 million video clips derived from more than 27.1k hours of high-resolution trailer videos. To annotate this data, we propose a systematic multimodal captioning framework. This framework first employs specialized unimodal models to extract descriptive features from multiple perspectives, including visual content, motion dynamics, and musical attributes (e.g., genre, instruments, mood). Subsequently, a large language model (LLM) is utilized to adaptively fuse these diverse descriptions into a single, coherent, and rich multimodal caption. This process yields VMChill-2M, a high-quality subset of 2 million clips with detailed multimodal annotations, and VMChill-Test, a manually refined test set for evaluation. We conduct extensive experiments on downstream tasks, including video understanding and generation, to establish benchmarks and demonstrate the dataset's quality. The results validate that VMChill effectively enhances model performance, highlighting its potential to facilitate future research in fine-grained multimodal learning. We will release the dataset, annotation codebase, and processing pipelines to support community research.

Zitieren

APA

Chi, X., Tian, Z., Chen, J., & Xue, W. (2026). VMChill: A Dataset for Fine-Grained Visual-Musical Synergy. Proceedings of the AAAI Conference on Artificial Intelligence, 40(5), 3353-3362. https://doi.org/10.1609/aaai.v40i5.37331

Vancouver

Chi X, Tian Z, Chen J, Xue W. VMChill: A Dataset for Fine-Grained Visual-Musical Synergy. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(5):3353-3362. doi:10.1609/aaai.v40i5.37331

BibTeX
@article{Chi2026,
  author = {Chi, Xiaowei and Tian, Zeyue and Chen, Jialiang and Xue, Wei},
  title = {VMChill: A Dataset for Fine-Grained Visual-Musical Synergy},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {5},
  pages = {3353--3362},
  doi = {10.1609/aaai.v40i5.37331}
}
Kostenlos starten →