EUVIMEDEUVIMED entdecken
Crossref

ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models

Yeji Park, Deokyeong Lee, Junsuk Choe, Buru Chang

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (6) · 6434-6442

DOI: 10.1609/aaai.v39i6.32689 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Yeji Park und 3 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Hallucinations in Multimodal Large Language Models (MLLMs) where generated responses fail to accurately reflect the given image pose a significant challenge to their reliability. To address this, we introduce ConVis, a novel training-free contrastive decoding method. ConVis leverages a text-to-image (T2I) generation model to semantically reconstruct the given image from hallucinated captions. By comparing the contrasting probability distributions produced by the original and reconstructed images, ConVis enables MLLMs to capture visual contrastive signals that penalize hallucination generation. Notably, this method operates purely within the decoding process, eliminating the need for additional data or model updates. Our extensive experiments on five popular benchmarks demonstrate that ConVis effectively reduces hallucinations across various MLLMs, highlighting its potential to enhance model reliability.

Zitieren

APA

Park, Y., Lee, D., Choe, J., & Chang, B. (2025). ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 39(6), 6434-6442. https://doi.org/10.1609/aaai.v39i6.32689

Vancouver

Park Y, Lee D, Choe J, Chang B. ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(6):6434-6442. doi:10.1609/aaai.v39i6.32689

BibTeX
@article{Park2025,
  author = {Park, Yeji and Lee, Deokyeong and Choe, Junsuk and Chang, Buru},
  title = {ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {6},
  pages = {6434--6442},
  doi = {10.1609/aaai.v39i6.32689}
}
Kostenlos starten →