Distillation-free Scaling of Large State-Space Models for Images and Videos
International Journal of Computer Vision · 2026-05 · 134 (5)
DOI: 10.1007/s11263-026-02824-0 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 17.04.2026. Verfasst von Hamid Suleman und 3 weiteren Autor:innen. Beteiligte Einrichtung: University of Bonn, Bonn, Germany. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 93 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Bundesministerium für Bildung und Forschung, European Research Council.
Abstract (Originalsprache)
State-space models (SSMs), exemplified by S4, have introduced a novel context modeling method by integrating state-space techniques into deep learning. Despite their effectiveness, SSMs struggle with global context modeling due to data-independent matrices. The Mamba model addresses this with data-dependent variants enabled by the S6 selective-scan algorithm, enhancing context modeling, especially for long sequences. However, Mamba-based architectures face significant parameter scalability challenges, limiting their utility in vision applications. This paper tackles the scalability issue of large SSMs for image classification and action recognition without relying on additional techniques like knowledge distillation. We analyze the distinct characteristics of Mamba-based and Attention-based models, proposing a Mamba-Attention interleaved architecture that enhances scalability, robustness, and performance. We demonstrate that the stable and efficient interleaved architecture resolves the scalability issue of Mamba-based architectures and increases robustness to common corruption artifacts. Our thorough evaluation on the ImageNet-1K, Kinetics-400, and Something-Something-v2 benchmarks demonstrates that our approach improves the accuracy of state-of-the-art Mamba-based architectures by up to $$+1.7$$ + 1.7 %.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- University of Bonn, Bonn, Germany.
- Lamarr Institute for Machine Learning and Artificial Intelligence, Dortmund, Germany.
- Department of Computer Science, Khalifa University, Abu Dhabi, United Arab Emirates.
ORCID-Profile: Hamid Suleman
Förderung
- Bundesministerium für Bildung und Forschung
- European Research Council
Zitieren
APA
Suleman, H., Wasim, S. T., Naseer, M., & Gall, J. (2026). Distillation-free Scaling of Large State-Space Models for Images and Videos. International Journal of Computer Vision, 134(5). https://doi.org/10.1007/s11263-026-02824-0
Vancouver
Suleman H, Wasim ST, Naseer M, Gall J. Distillation-free Scaling of Large State-Space Models for Images and Videos. International Journal of Computer Vision. 2026;134(5). doi:10.1007/s11263-026-02824-0
BibTeX
@article{Suleman2026,
author = {Suleman, Hamid and Wasim, Syed Talal and Naseer, Muzammal and Gall, Juergen},
title = {Distillation-free Scaling of Large State-Space Models for Images and Videos},
journal = {International Journal of Computer Vision},
year = {2026},
volume = {134},
number = {5},
doi = {10.1007/s11263-026-02824-0}
}
EUVIMED