EUVIMEDEUVIMED entdecken
Crossref

Long-form RewardBench: Evaluating Reward Models for Long-form Generation

Hui Huang, Yancheng He, Wei Liu, Muyun Yang, Jiaheng Liu, Kehai Chen, Bing Xu, Conghui Zhu, Hailong Cao, Tiejun Zhao

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (37) · 31149-31157

DOI: 10.1609/aaai.v40i37.40376 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Hui Huang und 9 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The widespread adoption of reinforcement learning-based alignment highlights the growing importance of reward models. Various benchmarks have been built to evaluate reward models in various domains and scenarios. However, a significant gap remains in assessing reward models for long-form generation, despite its critical role in real-world applications. To bridge this, we introduce Long-form RewardBench, the first reward modeling testbed specifically designed for long-form generation. Our benchmark encompasses five key subtasks: QA, RAG, Chat, Writing, and Reasoning. We collected instruction and preference data through a meticulously designed multi-stage data collection process, and conducted extensive experiments on 20+ mainstream reward models, including both classifiers and generative models. Our findings reveal that current models still lack long-form reward modeling capabilities. Furthermore, we designed a novel Long-form Needle-in-a-Haystack Test, which revealed a correlation between reward modeling performance and the error's position within a response, as well as the overall response length, with distinct characteristics observed between classification and generative models. Finally, we demonstrate that classifier exhibit better generalizability compared to generative models trained on the same data. As the first benchmark for long-form reward modeling, this work aims to offer a robust platform for visualizing progress in this crucial area.

Zitieren

APA

Huang, H., He, Y., Liu, W., Yang, M., Liu, J., Chen, K., Xu, B., Zhu, C., Cao, H., & Zhao, T. (2026). Long-form RewardBench: Evaluating Reward Models for Long-form Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37), 31149-31157. https://doi.org/10.1609/aaai.v40i37.40376

Vancouver

Huang H, He Y, Liu W, Yang M, Liu J, Chen K, et al. Long-form RewardBench: Evaluating Reward Models for Long-form Generation. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(37):31149-31157. doi:10.1609/aaai.v40i37.40376

BibTeX
@article{Huang2026,
  author = {Huang, Hui and He, Yancheng and Liu, Wei and Yang, Muyun and Liu, Jiaheng and Chen, Kehai and Xu, Bing and Zhu, Conghui and Cao, Hailong and Zhao, Tiejun},
  title = {Long-form RewardBench: Evaluating Reward Models for Long-form Generation},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {37},
  pages = {31149--31157},
  doi = {10.1609/aaai.v40i37.40376}
}
Kostenlos starten →