EUVIMEDEUVIMED entdecken
Crossref

Do Retrieval Augmented Language Models Know When They Don’t Know?

Youchao Zhou, Heyan Huang, Yicheng Liu, Rui Dai, Xinglin Wang, Xingchen Zhang, Shumin Shi, Yang Deng

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (41) · 35158-35166

DOI: 10.1609/aaai.v40i41.40822 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Youchao Zhou und 7 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Existing large language models (LLMs) occasionally generate plausible yet factually incorrect responses, known as hallucinations. Two main approaches have been proposed to mitigate hallucinations: retrieval-augmented language models (RALMs) and refusal post-training. However, current research predominantly focuses on their individual effectiveness while overlooking the evaluation of the refusal capability of RALMs. Ideally, if RALMs know when they do not know, they should refuse to answer. In this study, we ask the fundamental question: Do RALMs know when they don’t know? Specifically, we investigate three questions. First, are RALMs well calibrated with respect to different internal and external knowledge states? We examine the influence of various factors. Contrary to expectations, when all retrieved documents are irrelevant, RALMs still tend to refuse questions they could have answered correctly. Next, given the model's pronounced over-refusal behavior, we raise a second question: How does a RALM's refusal ability align with its calibration quality? Our results show that the over-refusal problem can be mitigated through in-context fine-tuning. However, we observe that improved refusal behavior does not necessarily imply better calibration or higher overall accuracy. Finally, we ask: Can we combine refusal-aware RALMs with uncertainty-based answer abstention to mitigate over-refusal? We develop a simple yet effective refusal mechanism for refusal-post-trained RALMs that improves their overall answer quality by balancing refusal and correct answers. Our study provides a more comprehensive understanding of the factors influencing RALM behavior. Meanwhile, we emphasize that uncertainty estimation for RALMs remains an open problem deserving deeper investigation.

Zitieren

APA

Zhou, Y., Huang, H., Liu, Y., Dai, R., Wang, X., Zhang, X., Shi, S., & Deng, Y. (2026). Do Retrieval Augmented Language Models Know When They Don’t Know?. Proceedings of the AAAI Conference on Artificial Intelligence, 40(41), 35158-35166. https://doi.org/10.1609/aaai.v40i41.40822

Vancouver

Zhou Y, Huang H, Liu Y, Dai R, Wang X, Zhang X, et al. Do Retrieval Augmented Language Models Know When They Don’t Know?. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(41):35158-35166. doi:10.1609/aaai.v40i41.40822

BibTeX
@article{Zhou2026,
  author = {Zhou, Youchao and Huang, Heyan and Liu, Yicheng and Dai, Rui and Wang, Xinglin and Zhang, Xingchen and Shi, Shumin and Deng, Yang},
  title = {Do Retrieval Augmented Language Models Know When They Don’t Know?},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {41},
  pages = {35158--35166},
  doi = {10.1609/aaai.v40i41.40822}
}
Kostenlos starten →