Causal-HalBench: Uncovering LVLMs Object Hallucinations Through Causal Intervention
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 34169-34177
DOI: 10.1609/aaai.v40i40.40712 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zhe Xu und 4 weiteren Autor:innen.
Abstract (Originalsprache)
Large Vision-Language Models (LVLMs) often suffer from object hallucination, making erroneous judgments about the presence of objects in images. We propose this primarily stems from spurious correlations arising when models strongly associate highly co-occurring objects during training, leading to hallucinated objects influenced by visual context. Current benchmarks mainly focus on hallucination detection but lack a formal characterization and quantitative evaluation of spurious correlations in LVLMs. To address this, we introduce causal analysis into the object recognition scenario of LVLMs, establishing a Structural Causal Model (SCM). Utilizing the language of causality, we formally define spurious correlations arising from co-occurrence bias. To quantify the influence induced by these spurious correlations, we develop Causal-HalBench, a benchmark specifically constructed with counterfactual samples and integrated with comprehensive causal metrics designed to assess model robustness against spurious correlations. Concurrently, we propose an extensible pipeline for the construction of these counterfactual samples, leveraging the capabilities of proprietary LVLMs and Text-to-Image (T2I) models for their generation. Our evaluations on mainstream LVLMs using Causal-HalBench demonstrate these models exhibit susceptibility to spurious correlations, albeit to varying extents.
Zitieren
APA
Xu, Z., Wang, Z., Wu, J., Lu, J., & Wang, X. (2026). Causal-HalBench: Uncovering LVLMs Object Hallucinations Through Causal Intervention. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 34169-34177. https://doi.org/10.1609/aaai.v40i40.40712
Vancouver
Xu Z, Wang Z, Wu J, Lu J, Wang X. Causal-HalBench: Uncovering LVLMs Object Hallucinations Through Causal Intervention. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):34169-34177. doi:10.1609/aaai.v40i40.40712
BibTeX
@article{Xu2026,
author = {Xu, Zhe and Wang, Zhicai and Wu, Junkang and Lu, Jinda and Wang, Xiang},
title = {Causal-HalBench: Uncovering LVLMs Object Hallucinations Through Causal Intervention},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {40},
pages = {34169--34177},
doi = {10.1609/aaai.v40i40.40712}
}
EUVIMED