Monitoring Before Answering: Epistemic Monitoring Improves Knowledge-Limit Recognition in Large Language Models
Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 57-61
DOI: 10.1609/aaaiss.v10i1.43391 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Malia Barker und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Large language models (LLMs) need to recognize not only how to answer a question, but also when their knowledge is insufficient to provide a reliable answer. We investigate whether prompting a model to evaluate the available evidence and its knowledge limitations before answering improves this ability. Inspired by higher-order theories of consciousness, this approach makes properties of a prospective response available for evaluation before the final answer is selected. It therefore differs from self-reflection approaches that ask a model to reconsider an answer it has already produced. To the best of our knowledge, this is the first study to use this higher-order framework to examine pre-answer epistemic monitoring in LLMs. We evaluate a 120-billion-parameter language model on a balanced sample of 300 questions from MetaMedQA. We compare a direct answer-only prompt, which passes the question and multiple-choice options to the model, with two monitoring prompts that additionally ask the model to evaluate epistemic support. Exact answer accuracy increases from 56.7% with direct prompting to 69.3% with minimal monitoring and 73.0% with structured monitoring. This improvement is concentrated in questions that exceed the model's knowledge: recall for the "I don't know or cannot answer" option increases from 0% to 40% and 54%, while accuracy on ordinary questions remains approximately stable. The structured assessments also influence the model's final response: whenever the model identifies a blocking problem, it selects a non-substantive response. These results show that evaluating epistemic support before answering substantially improves the recognition of knowledge limitations, while explicitly structuring this evaluation provides a smaller additional benefit.
Zitieren
APA
Barker, M., Mehrotra, P., Ferrier, E., & Serra, E. (2026). Monitoring Before Answering: Epistemic Monitoring Improves Knowledge-Limit Recognition in Large Language Models. Proceedings of the AAAI Symposium Series, 10(1), 57-61. https://doi.org/10.1609/aaaiss.v10i1.43391
Vancouver
Barker M, Mehrotra P, Ferrier E, Serra E. Monitoring Before Answering: Epistemic Monitoring Improves Knowledge-Limit Recognition in Large Language Models. Proceedings of the AAAI Symposium Series. 2026;10(1):57-61. doi:10.1609/aaaiss.v10i1.43391
BibTeX
@article{Barker2026,
author = {Barker, Malia and Mehrotra, Parth and Ferrier, Edward and Serra, Edoardo},
title = {Monitoring Before Answering: Epistemic Monitoring Improves Knowledge-Limit Recognition in Large Language Models},
journal = {Proceedings of the AAAI Symposium Series},
year = {2026},
volume = {10},
number = {1},
pages = {57--61},
doi = {10.1609/aaaiss.v10i1.43391}
}
EUVIMED