WildVideo: Benchmarking LMMs for Understanding Video-Language Interaction
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025
DOI: 10.1109/tpami.2025.3592831 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. Verfasst von Songyuan Yang und 6 weiteren Autor:innen. Hauptthemen laut MEDLINE: Video Recording, Language, Image Processing, Computer-Assisted. Die Arbeit nennt 70 Literaturangaben. Gefördert von: Young Scientists Fund of the Hunan Natural Science Foundation, Youth Independent Innovation Science Fund Project of NUDT, National Natural Science Foundation of China.
Abstract (Originalsprache)
We introduce WildVideo, an open-world benchmark dataset designed to address how to assess hallucination of Large Multi-modal Models (LMMs) for understanding video-language interaction in the wild. Our WildVideo comprehensively tests the perceptual, cognitive, and contextual comprehension hallucination of LMMs through both single-turn and multi-turn open-ended question-answering (QA) tasks on videos captured from two human perspectives (i.e. first-person view and third-person view). We define 9 distinct tasks that challenge LMMs across multi-level perceptual tasks (e.g., static and dynamic perception), multi-aspect cognitive tasks (e.g., commonsense, world knowledge), and multi-faceted contextual comprehension tasks (e.g., contextual ellipsis, cross-turn retrieval). The benchmark consists of 1,318 meticulously curated videos, supplemented with 13,704 single-turn QA pairs and 1,585 multi-turn dialogues (up to 5 turns). We evaluated 14 commonly-used LMMs on WildVideo, revealing significant hallucination issues of current LMMs, highlighting substantial gaps in their current capabilities.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- Young Scientists Fund of the Hunan Natural Science Foundation
- Youth Independent Innovation Science Fund Project of NUDT
- National Natural Science Foundation of China
Zitieren
APA
Yang, S., Yu, W., Yang, W., Liu, X., Tan, H., Lan, L., & Xiao, N. (2025). WildVideo: Benchmarking LMMs for Understanding Video-Language Interaction. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2025.3592831
Vancouver
Yang S, Yu W, Yang W, Liu X, Tan H, Lan L, et al. WildVideo: Benchmarking LMMs for Understanding Video-Language Interaction. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025. doi:10.1109/tpami.2025.3592831
BibTeX
@article{Yang2025,
author = {Yang, Songyuan and Yu, Weijiang and Yang, Wenjing and Liu, Xinwang and Tan, Huibin and Lan, Long and Xiao, Nong},
title = {WildVideo: Benchmarking LMMs for Understanding Video-Language Interaction},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
doi = {10.1109/tpami.2025.3592831}
}
EUVIMED