SpaRRTa: A Synthetic Benchmark for Evaluating Spatial Intelligence in Visual Foundation Models
International Journal of Computer Vision · 2026-12 · 134 (12)
DOI: 10.1007/s11263-026-03112-7 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 13.11.2026. Verfasst von Turhan Can Kargin und 4 weiteren Autor:innen. Die Arbeit nennt 89 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Narodowe Centrum Nauki.
Abstract (Originalsprache)
Visual Foundation Models (VFMs), such as DINO and CLIP, excel in semantic understanding of images but exhibit limited spatial reasoning capabilities, which limits their applicability to embodied systems. As a result, recent work has incorporated some 3D tasks (such as depth estimation) into VFM training. However, VFM performance remains inconsistent across other spatial tasks, raising the question of whether these models truly possess spatial awareness or overfit to specific 3D objectives. To address this question, we introduce the Spatial Relation Recognition Task (SpaRRTa) benchmark, which evaluates the ability of VFMs to identify static directional relations between visible objects. Unlike traditional 3D objectives that focus on precise metric prediction (e.g., surface normal estimation), SpaRRTa probes a fundamental capability underpinning more advanced forms of human-like spatial understanding. SpaRRTa generates an arbitrary number of photorealistic images with diverse scenes and fully controllable object arrangements, along with freely accessible spatial annotations. Evaluating a range of state-of-the-art VFMs, we reveal significant disparities in their ability to encode directional spatial relations. Through our analysis, we provide insights into the mechanisms that support or hinder this form of spatial awareness in modern VFMs. We hope that SpaRRTa will serve as a useful tool for guiding the development of future spatially aware visual models. Our project page can be found at https://sparrta.gmum.net/ .
Förderung
- Narodowe Centrum Nauki
Zitieren
APA
Kargin, T. C., Jasiński, W., Pardyl, A., Zieliński, B., & Przewiȩźlikowski, M. (2026). SpaRRTa: A Synthetic Benchmark for Evaluating Spatial Intelligence in Visual Foundation Models. International Journal of Computer Vision, 134(12). https://doi.org/10.1007/s11263-026-03112-7
Vancouver
Kargin TC, Jasiński W, Pardyl A, Zieliński B, Przewiȩźlikowski M. SpaRRTa: A Synthetic Benchmark for Evaluating Spatial Intelligence in Visual Foundation Models. International Journal of Computer Vision. 2026;134(12). doi:10.1007/s11263-026-03112-7
BibTeX
@article{Kargin2026,
author = {Kargin, Turhan Can and Jasiński, Wojciech and Pardyl, Adam and Zieliński, Bartosz and Przewiȩźlikowski, Marcin},
title = {SpaRRTa: A Synthetic Benchmark for Evaluating Spatial Intelligence in Visual Foundation Models},
journal = {International Journal of Computer Vision},
year = {2026},
volume = {134},
number = {12},
doi = {10.1007/s11263-026-03112-7}
}
EUVIMED