BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generation
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 33737-33745
DOI: 10.1609/aaai.v40i40.40664 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yuhao Wang und 6 weiteren Autor:innen.
Abstract (Originalsprache)
With the rapid advancement of large language models (LLMs), retrieval-augmented generation (RAG) has emerged as a critical approach to supplement the inherent knowledge limitations of LLMs. However, due to the typically large volume of retrieved information, RAG tends to operate with long context lengths. From the perspective of entropy engineering, we identify unconstrained entropy growth and attention dilution due to long retrieval context as significant factors affecting RAG performance. In this paper, we propose the balanced entropy-engineered RAG (BEE-RAG) framework, which improves the adaptability of RAG systems to varying context lengths through the principle of entropy invariance. By leveraging balanced context entropy to reformulate attention dynamics, BEE-RAG separates attention sensitivity from context length, ensuring a stable entropy level. Building upon this, we introduce a zero-shot inference strategy for multi-importance estimation and a parameter-efficient adaptive fine-tuning mechanism to obtain the optimal balancing factor for different settings. Extensive experiments across multiple RAG tasks demonstrate the effectiveness of BEE-RAG.
Zitieren
APA
Wang, Y., Ren, R., Wang, Y., Liu, J., Zhao, X., Wu, H., & Wang, H. (2026). BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 33737-33745. https://doi.org/10.1609/aaai.v40i40.40664
Vancouver
Wang Y, Ren R, Wang Y, Liu J, Zhao X, Wu H, et al. BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generation. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):33737-33745. doi:10.1609/aaai.v40i40.40664
BibTeX
@article{Wang2026,
author = {Wang, Yuhao and Ren, Ruiyang and Wang, Yucheng and Liu, Jing and Zhao, Xin and Wu, Hua and Wang, Haifeng},
title = {BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generation},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {40},
pages = {33737--33745},
doi = {10.1609/aaai.v40i40.40664}
}
EUVIMED