EUVIMEDEUVIMED entdecken
Crossref

Thinking Forward and Backward: Multi-Objective Reinforcement Learning for Retrieval-Augmented Reasoning

Wenda Wei, Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Lixin Su, Shuaiqiang Wang, Dawei Yin, Maarten de Rijke, Xueqi Cheng

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 33836-33844

DOI: 10.1609/aaai.v40i40.40675 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Wenda Wei und 8 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Retrieval-augmented generation (RAG) has proven to be effective in mitigating hallucinations in large language models, yet its effectiveness remains limited in complex, multi-step reasoning scenarios. Recent efforts have incorporated search-based interactions into RAG, enabling iterative reasoning with real-time retrieval. Most approaches rely on outcome-based supervision, offering no explicit guidance for intermediate steps. This often leads to reward hacking and degraded response quality. We propose Bi-RAR, a novel retrieval-augmented reasoning framework that evaluates each intermediate step jointly in both forward and backward directions. To assess the information completeness of each step, we introduce a bidirectional information distance grounded in Kolmogorov complexity, approximated via language model generation probabilities. This quantification measures both how far the current reasoning is from the answer and how well it addresses the question. To optimize reasoning under these bidirectional signals, we adopt a multi-objective reinforcement learning framework with a cascading reward structure that emphasizes early trajectory alignment. Empirical results on seven question answering benchmarks demonstrate that Bi-RAR surpasses previous methods and enables efficient interaction and reasoning with the search engine during training and inference.

Zitieren

APA

Wei, W., Liu, Y., Zhang, R., Guo, J., Su, L., Wang, S., Yin, D., Rijke, M. D., & Cheng, X. (2026). Thinking Forward and Backward: Multi-Objective Reinforcement Learning for Retrieval-Augmented Reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 33836-33844. https://doi.org/10.1609/aaai.v40i40.40675

Vancouver

Wei W, Liu Y, Zhang R, Guo J, Su L, Wang S, et al. Thinking Forward and Backward: Multi-Objective Reinforcement Learning for Retrieval-Augmented Reasoning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):33836-33844. doi:10.1609/aaai.v40i40.40675

BibTeX
@article{Wei2026,
  author = {Wei, Wenda and Liu, Yu-An and Zhang, Ruqing and Guo, Jiafeng and Su, Lixin and Wang, Shuaiqiang and Yin, Dawei and Rijke, Maarten de and Cheng, Xueqi},
  title = {Thinking Forward and Backward: Multi-Objective Reinforcement Learning for Retrieval-Augmented Reasoning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {40},
  pages = {33836--33844},
  doi = {10.1609/aaai.v40i40.40675}
}
Kostenlos starten →