View-on-Graph: Zero-Shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (9) · 7386-7394
DOI: 10.1609/aaai.v40i9.37677 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yuanyuan Liu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref).
Abstract (Originalsprache)
3D visual grounding (3DVG) identifies objects in 3D scenes from language descriptions. Existing zero-shot approaches leverage 2D vision–language models (VLMs) by converting 3D spatial information (SI) into forms amenable to VLM processing, typically as composite inputs such as specified-view renderings or video sequences with overlaid object markers. However, this VLM ⊕ SI paradigm yields entangled visual representations that compel the VLM to process entire cluttered cues, making it hard to exploit spatial–semantic relationships effectively. In this work, we propose a new VLM ⊗ SI paradigm that externalizes the 3D SI into a form enabling the VLM to incrementally retrieve only what it needs during reasoning. We instantiate this paradigm with a novel View-on-Graph (VoG) method, which organizes the scene into a multi-modal, multi-layer scene graph and allows the VLM to operate as an active agent that selectively accesses necessary cues as it traverses the scene. This design offers two intrinsic advantages: (i) by structuring 3D context into a spatially and semantically coherent scene graph rather than confounding the VLM with densely entangled visual inputs, it lowers the VLM's reasoning difficulty; and (ii) by actively exploring and reasoning over the scene graph, it naturally produces transparent, step-by-step traces for interpretable 3DVG. Extensive experiments show that VoG achieves state-of-the-art zero-shot performance, establishing structured scene exploration as a promising strategy for advancing zero-shot 3DVG.
Zitieren
APA
Liu, Y., Mei, H., Zhan, D., Zhao, J., Zhou, D., Dong, B., & Yang, X. (2026). View-on-Graph: Zero-Shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 40(9), 7386-7394. https://doi.org/10.1609/aaai.v40i9.37677
Vancouver
Liu Y, Mei H, Zhan D, Zhao J, Zhou D, Dong B, et al. View-on-Graph: Zero-Shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(9):7386-7394. doi:10.1609/aaai.v40i9.37677
BibTeX
@article{Liu2026,
author = {Liu, Yuanyuan and Mei, Haiyang and Zhan, Dongyang and Zhao, Jiayue and Zhou, Dongsheng and Dong, Bo and Yang, Xin},
title = {View-on-Graph: Zero-Shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {9},
pages = {7386--7394},
doi = {10.1609/aaai.v40i9.37677}
}
EUVIMED