Can You Trust What I Think? Analyzing and Improving Verbalized Uncertainty and Factuality in Reasoning-Based Large Language Models
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (48) · 41534-41536
DOI: 10.1609/aaai.v40i48.42331 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Tianruo Rose Xu.
Abstract (Originalsprache)
Reasoning-based large language models now often produce natural-language thinking traces alongside their answers, but it remains unclear whether these verbalized uncertainties faithfully reflect their knowledge or can be used to improve factuality. We study this question for long-form, knowledge-intensive biography generation. Our pipeline decomposes thinking traces and responses into atomic facts, filters out planning-style content, labels factual reasoning by certainty, and aligns response facts to their supporting reasoning, enabling plan-based filtering, self-verification, and a classifier that predicts factuality from facts and associated reasoning. Preliminary results suggest that high-certainty reasoning is more likely to be included and correct and that structured use of these signals can improve factual precision, though broader validation across models and dataset will be needed.
Zitieren
APA
Xu, T. R. (2026). Can You Trust What I Think? Analyzing and Improving Verbalized Uncertainty and Factuality in Reasoning-Based Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(48), 41534-41536. https://doi.org/10.1609/aaai.v40i48.42331
Vancouver
Xu TR. Can You Trust What I Think? Analyzing and Improving Verbalized Uncertainty and Factuality in Reasoning-Based Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(48):41534-41536. doi:10.1609/aaai.v40i48.42331
BibTeX
@article{Xu2026,
author = {Xu, Tianruo Rose},
title = {Can You Trust What I Think? Analyzing and Improving Verbalized Uncertainty and Factuality in Reasoning-Based Large Language Models},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {48},
pages = {41534--41536},
doi = {10.1609/aaai.v40i48.42331}
}
EUVIMED