Mitigating Hallucinations in Large Language Models via Contrastive Decoding and Entropy Minimization
International Journal of Computational and Biological Sciences · 31.01.2026 · 3 (1) · 17-25
DOI: 10.66238/ijcbs39 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computational and Biological Sciences, online erschienen am 31.01.2026. Verfasst von Ji-Hye Song und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The proliferation of Large Language Models has revolutionized natural language processing, yet the persistent issue of hallucination—where models generate plausible but factually incorrect information—remains a critical impediment to their deployment in high-stakes environments. This paper introduces a novel decoding framework that synergizes Contrastive Decoding with Entropy Minimization to enhance factual reliability without sacrificing linguistic fluency. While traditional decoding strategies such as nucleus sampling rely heavily on probability truncation, they often fail to distinguish between high probability generic responses and contextually accurate specifics. Our approach leverages a dual-model system, pitting an expert model against an amateur model to penalize tokens that are statistically probable but conceptually vacuous, while simultaneously employing an entropy-based regularization term to suppress generation during states of high uncertainty. Through extensive experimentation on standard factual benchmarks, we demonstrate that this composite method significantly reduces hallucination rates compared to state-of-the-art baselines. The results suggest that addressing the stochastic flaws of auto-regressive generation requires not only better training data but also more rigorous, inference-time intervention mechanisms that account for the semantic entropy of the output distribution.
Zitieren
APA
Song, J., & Montgomery, S. L. (2026). Mitigating Hallucinations in Large Language Models via Contrastive Decoding and Entropy Minimization. International Journal of Computational and Biological Sciences, 3(1), 17-25. https://doi.org/10.66238/ijcbs39
Vancouver
Song J, Montgomery SL. Mitigating Hallucinations in Large Language Models via Contrastive Decoding and Entropy Minimization. International Journal of Computational and Biological Sciences. 2026;3(1):17-25. doi:10.66238/ijcbs39
BibTeX
@article{Song2026,
author = {Song, Ji-Hye and Montgomery, Sarah L.},
title = {Mitigating Hallucinations in Large Language Models via Contrastive Decoding and Entropy Minimization},
journal = {International Journal of Computational and Biological Sciences},
year = {2026},
volume = {3},
number = {1},
pages = {17--25},
doi = {10.66238/ijcbs39}
}
EUVIMED