EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Rethinking Visual Attention for Reducing Hallucination in Large Vision–Language Models

Xuewen Li, Yuan Liu

Applied Sciences · 23.04.2026 · 16 (9) · 4143

DOI: 10.3390/app16094143 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 23.04.2026. Verfasst von Xuewen Li und 1 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Vision–Language Models (LVLMs) have achieved strong performance in multimodal understanding and generation. However, they remain prone to hallucination, where generated content deviates from the visual input, reducing output reliability. We analyze the attention mechanism and identify two key issues in visual information use. The model exhibits insufficient overall attention to visual tokens and weak or dispersed attention to semantically relevant regions, limiting effective visual grounding. We propose a tuning-free attention intervention method applied at inference time. In the encoding stage, we apply a structured rescaling to the attention logits associated with visual tokens, introducing a structural bias in the visual subspace. In the decoding stage, we filter attention heads based on their response magnitudes and perform weighted aggregation using their global response intensities. This design reinforces salient visual evidence while suppressing weak or diffuse attention patterns. Experiments on CHAIR and POPE show that our method reduces hallucination without additional training. On the CHAIR benchmark, it reduces the sentence-level metric by 15.5% and the instance-level metric by 5.7% on average, while consistently improving performance across multiple LVLMs and maintaining strong results on general multimodal benchmarks such as MME.

Zitieren

APA

Li, X., & Liu, Y. (2026). Rethinking Visual Attention for Reducing Hallucination in Large Vision–Language Models. Applied Sciences, 16(9), 4143. https://doi.org/10.3390/app16094143

Vancouver

Li X, Liu Y. Rethinking Visual Attention for Reducing Hallucination in Large Vision–Language Models. Applied Sciences. 2026;16(9):4143. doi:10.3390/app16094143

BibTeX
@article{Li2026,
  author = {Li, Xuewen and Liu, Yuan},
  title = {Rethinking Visual Attention for Reducing Hallucination in Large Vision–Language Models},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {9},
  pages = {4143},
  doi = {10.3390/app16094143}
}
Kostenlos starten →