EUVIMEDEUVIMED entdecken
Crossref

State-Space Hierarchical Compression with Gated Attention and Learnable Sampling for Hour-Long Video Understanding in Large Multimodal Models

Geewook Kim, Minjoon Seo

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (7) · 5656-5664

DOI: 10.1609/aaai.v40i7.37485 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Geewook Kim und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We propose an efficient framework to compress massive video-frame features before feeding them into large multimodal models, thereby mitigating the severe token explosion arising from hour-long videos. Our design leverages a bidirectional state-space model equipped with a gated skip connection and a learnable weighted-average pooling mechanism applied to periodically inserted learned queries. This structure enables hierarchical downsampling across both spatial and temporal dimensions, preserving performance in a cost-effective manner. Across challenging hour-long video understanding tasks, our approach demonstrates competitive results against state-of-the-art models, while significantly reducing overall token budget. Notably, replacing our state-space model with conventional modules results in substantial performance degradation, highlighting the advantages of the proposed state-space modeling for effectively compressing multi-frame video information. Our framework emphasizes resource-conscious efficiency, making it practical for real-world deployments. We validate its scalability and generality across multiple benchmarks, achieving the dual objectives of efficient resource usage and comprehensive video understanding.

Zitieren

APA

Kim, G., & Seo, M. (2026). State-Space Hierarchical Compression with Gated Attention and Learnable Sampling for Hour-Long Video Understanding in Large Multimodal Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(7), 5656-5664. https://doi.org/10.1609/aaai.v40i7.37485

Vancouver

Kim G, Seo M. State-Space Hierarchical Compression with Gated Attention and Learnable Sampling for Hour-Long Video Understanding in Large Multimodal Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(7):5656-5664. doi:10.1609/aaai.v40i7.37485

BibTeX
@article{Kim2026,
  author = {Kim, Geewook and Seo, Minjoon},
  title = {State-Space Hierarchical Compression with Gated Attention and Learnable Sampling for Hour-Long Video Understanding in Large Multimodal Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {7},
  pages = {5656--5664},
  doi = {10.1609/aaai.v40i7.37485}
}
Kostenlos starten →