Rethinking Visual Attention for Reducing Hallucination in Large Vision–Language Models
Applied Sciences · 23.04.2026 · 16 (9) · 4143
DOI: 10.3390/app16094143 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 23.04.2026. Verfasst von Xuewen Li und 1 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Large Vision–Language Models (LVLMs) have achieved strong performance in multimodal understanding and generation. However, they remain prone to hallucination, where generated content deviates from the visual input, reducing output reliability. We analyze the attention mechanism and identify two key issues in visual information use. The model exhibits insufficient overall attention to visual tokens and weak or dispersed attention to semantically relevant regions, limiting effective visual grounding. We propose a tuning-free attention intervention method applied at inference time. In the encoding stage, we apply a structured rescaling to the attention logits associated with visual tokens, introducing a structural bias in the visual subspace. In the decoding stage, we filter attention heads based on their response magnitudes and perform weighted aggregation using their global response intensities. This design reinforces salient visual evidence while suppressing weak or diffuse attention patterns. Experiments on CHAIR and POPE show that our method reduces hallucination without additional training. On the CHAIR benchmark, it reduces the sentence-level metric by 15.5% and the instance-level metric by 5.7% on average, while consistently improving performance across multiple LVLMs and maintaining strong results on general multimodal benchmarks such as MME.
Zitieren
APA
Li, X., & Liu, Y. (2026). Rethinking Visual Attention for Reducing Hallucination in Large Vision–Language Models. Applied Sciences, 16(9), 4143. https://doi.org/10.3390/app16094143
Vancouver
Li X, Liu Y. Rethinking Visual Attention for Reducing Hallucination in Large Vision–Language Models. Applied Sciences. 2026;16(9):4143. doi:10.3390/app16094143
BibTeX
@article{Li2026,
author = {Li, Xuewen and Liu, Yuan},
title = {Rethinking Visual Attention for Reducing Hallucination in Large Vision–Language Models},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {9},
pages = {4143},
doi = {10.3390/app16094143}
}
EUVIMED