EUVIMEDEUVIMED entdecken
Crossref

APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrieval

Hong Gao, Yiming Bao, Xuezhen Tu, Bin Zhong, Linan Yue, Min-Ling Zhang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (6) · 4113-4121

DOI: 10.1609/aaai.v40i6.42406 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Hong Gao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Current multimodal large language models (MLLMs) struggle with hour-level video understanding, facing significant challenges not only in modeling the substantial information volume of long videos but also in overcoming the memory wall and resource constraints during both training and inference. Although recent training-free approaches have alleviated resource demands by compressing visual features, their reliance on incomplete visual information limits the performance potential. To address these limitations, we propose Adaptive Pivot Visual information Retrieval (APVR), a training-free framework that hierarchically retrieves and retains sufficient and important visual information. It breakthroughs the memory wall limitation via two complementary components: Pivot Frame Retrieval employs query expansion and iterative spatio-semantic confidence scoring to identify relevant video frames, and Pivot Token Retrieval performs query-aware attention-driven token selection within up to 1024 pivot frames. This dual granularity approach enables the processing of hour-long videos while maintaining semantic fidelity. Experimental validations on three different baseline MLLMs demonstrate significant performance improvements up to 9.5%, 4.6% and 9.7% on LongVideoBench, VideoMME and MLVU, respectively. APVR achieves state-of-the-art results for both training-free and training-based approaches.

Zitieren

APA

Gao, H., Bao, Y., Tu, X., Zhong, B., Yue, L., & Zhang, M. (2026). APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, 40(6), 4113-4121. https://doi.org/10.1609/aaai.v40i6.42406

Vancouver

Gao H, Bao Y, Tu X, Zhong B, Yue L, Zhang M. APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(6):4113-4121. doi:10.1609/aaai.v40i6.42406

BibTeX
@article{Gao2026,
  author = {Gao, Hong and Bao, Yiming and Tu, Xuezhen and Zhong, Bin and Yue, Linan and Zhang, Min-Ling},
  title = {APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrieval},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {6},
  pages = {4113--4121},
  doi = {10.1609/aaai.v40i6.42406}
}
Kostenlos starten →