EUVIMEDEUVIMED entdecken
Crossref

WildVideo: Benchmarking LMMs for Understanding Video-Language Interaction

Songyuan Yang, Weijiang Yu, Wenjing Yang, Xinwang Liu, Huibin Tan, Long Lan, Nong Xiao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025

DOI: 10.1109/tpami.2025.3592831 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. Verfasst von Songyuan Yang und 6 weiteren Autor:innen. Hauptthemen laut MEDLINE: Video Recording, Language, Image Processing, Computer-Assisted. Die Arbeit nennt 70 Literaturangaben. Gefördert von: Young Scientists Fund of the Hunan Natural Science Foundation, Youth Independent Innovation Science Fund Project of NUDT, National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We introduce WildVideo, an open-world benchmark dataset designed to address how to assess hallucination of Large Multi-modal Models (LMMs) for understanding video-language interaction in the wild. Our WildVideo comprehensively tests the perceptual, cognitive, and contextual comprehension hallucination of LMMs through both single-turn and multi-turn open-ended question-answering (QA) tasks on videos captured from two human perspectives (i.e. first-person view and third-person view). We define 9 distinct tasks that challenge LMMs across multi-level perceptual tasks (e.g., static and dynamic perception), multi-aspect cognitive tasks (e.g., commonsense, world knowledge), and multi-faceted contextual comprehension tasks (e.g., contextual ellipsis, cross-turn retrieval). The benchmark consists of 1,318 meticulously curated videos, supplemented with 13,704 single-turn QA pairs and 1,585 multi-turn dialogues (up to 5 turns). We evaluated 14 commonly-used LMMs on WildVideo, revealing significant hallucination issues of current LMMs, highlighting substantial gaps in their current capabilities.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Video RecordingLanguageImage Processing, Computer-AssistedHumansBenchmarkingDatabases, FactualComprehension

Förderung

  • Young Scientists Fund of the Hunan Natural Science Foundation
  • Youth Independent Innovation Science Fund Project of NUDT
  • National Natural Science Foundation of China

Zitieren

APA

Yang, S., Yu, W., Yang, W., Liu, X., Tan, H., Lan, L., & Xiao, N. (2025). WildVideo: Benchmarking LMMs for Understanding Video-Language Interaction. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2025.3592831

Vancouver

Yang S, Yu W, Yang W, Liu X, Tan H, Lan L, et al. WildVideo: Benchmarking LMMs for Understanding Video-Language Interaction. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025. doi:10.1109/tpami.2025.3592831

BibTeX
@article{Yang2025,
  author = {Yang, Songyuan and Yu, Weijiang and Yang, Wenjing and Liu, Xinwang and Tan, Huibin and Lan, Long and Xiao, Nong},
  title = {WildVideo: Benchmarking LMMs for Understanding Video-Language Interaction},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  doi = {10.1109/tpami.2025.3592831}
}
Kostenlos starten →