RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
ACM Computing Surveys · 31.01.2026 · 58 (2) · 1-37
DOI: 10.1145/3743127 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 10.09.2025. Verfasst von Shreyas Chaudhari und 7 weiteren Autor:innen. Die Arbeit wurde bisher 96-mal zitiert (Crossref) und nennt 222 Literaturangaben.
Abstract (Originalsprache)
A significant challenge in training large language models (LLMs) as effective assistants is aligning them with human preferences. Reinforcement learning from human feedback (RLHF) has emerged as a promising solution. However, our understanding of RLHF is often limited to initial design choices. This article analyzes RLHF through reinforcement learning principles, focusing on the reward model. It examines modeling choices and function approximation caveats, highlighting assumptions about reward expressivity and revealing limitations like incorrect generalization, model misspecification, and sparse feedback. A categorical review of current literature provides insights for researchers to understand the challenges of RLHF and build upon existing methods.
Zitieren
APA
Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Silva, B. C. D. (2026). RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs. ACM Computing Surveys, 58(2), 1-37. https://doi.org/10.1145/3743127
Vancouver
Chaudhari S, Aggarwal P, Murahari V, Rajpurohit T, Kalyan A, Narasimhan K, et al. RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs. ACM Computing Surveys. 2026;58(2):1-37. doi:10.1145/3743127
BibTeX
@article{Chaudhari2026,
author = {Chaudhari, Shreyas and Aggarwal, Pranjal and Murahari, Vishvak and Rajpurohit, Tanmay and Kalyan, Ashwin and Narasimhan, Karthik and Deshpande, Ameet and Silva, Bruno Castro da},
title = {RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs},
journal = {ACM Computing Surveys},
year = {2026},
volume = {58},
number = {2},
pages = {1--37},
doi = {10.1145/3743127}
}
EUVIMED