EUVIMEDEUVIMED entdecken
Crossref

Causal-HalBench: Uncovering LVLMs Object Hallucinations Through Causal Intervention

Zhe Xu, Zhicai Wang, Junkang Wu, Jinda Lu, Xiang Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 34169-34177

DOI: 10.1609/aaai.v40i40.40712 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zhe Xu und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Vision-Language Models (LVLMs) often suffer from object hallucination, making erroneous judgments about the presence of objects in images. We propose this primarily stems from spurious correlations arising when models strongly associate highly co-occurring objects during training, leading to hallucinated objects influenced by visual context. Current benchmarks mainly focus on hallucination detection but lack a formal characterization and quantitative evaluation of spurious correlations in LVLMs. To address this, we introduce causal analysis into the object recognition scenario of LVLMs, establishing a Structural Causal Model (SCM). Utilizing the language of causality, we formally define spurious correlations arising from co-occurrence bias. To quantify the influence induced by these spurious correlations, we develop Causal-HalBench, a benchmark specifically constructed with counterfactual samples and integrated with comprehensive causal metrics designed to assess model robustness against spurious correlations. Concurrently, we propose an extensible pipeline for the construction of these counterfactual samples, leveraging the capabilities of proprietary LVLMs and Text-to-Image (T2I) models for their generation. Our evaluations on mainstream LVLMs using Causal-HalBench demonstrate these models exhibit susceptibility to spurious correlations, albeit to varying extents.

Zitieren

APA

Xu, Z., Wang, Z., Wu, J., Lu, J., & Wang, X. (2026). Causal-HalBench: Uncovering LVLMs Object Hallucinations Through Causal Intervention. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 34169-34177. https://doi.org/10.1609/aaai.v40i40.40712

Vancouver

Xu Z, Wang Z, Wu J, Lu J, Wang X. Causal-HalBench: Uncovering LVLMs Object Hallucinations Through Causal Intervention. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):34169-34177. doi:10.1609/aaai.v40i40.40712

BibTeX
@article{Xu2026,
  author = {Xu, Zhe and Wang, Zhicai and Wu, Junkang and Lu, Jinda and Wang, Xiang},
  title = {Causal-HalBench: Uncovering LVLMs Object Hallucinations Through Causal Intervention},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {40},
  pages = {34169--34177},
  doi = {10.1609/aaai.v40i40.40712}
}
Kostenlos starten →