EUVIMEDEUVIMED entdecken
Crossref

RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs

Shreyas Chaudhari, Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan, Ameet Deshpande, Bruno Castro da Silva

ACM Computing Surveys · 31.01.2026 · 58 (2) · 1-37

DOI: 10.1145/3743127 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 10.09.2025. Verfasst von Shreyas Chaudhari und 7 weiteren Autor:innen. Die Arbeit wurde bisher 96-mal zitiert (Crossref) und nennt 222 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A significant challenge in training large language models (LLMs) as effective assistants is aligning them with human preferences. Reinforcement learning from human feedback (RLHF) has emerged as a promising solution. However, our understanding of RLHF is often limited to initial design choices. This article analyzes RLHF through reinforcement learning principles, focusing on the reward model. It examines modeling choices and function approximation caveats, highlighting assumptions about reward expressivity and revealing limitations like incorrect generalization, model misspecification, and sparse feedback. A categorical review of current literature provides insights for researchers to understand the challenges of RLHF and build upon existing methods.

Zitieren

APA

Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Silva, B. C. D. (2026). RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs. ACM Computing Surveys, 58(2), 1-37. https://doi.org/10.1145/3743127

Vancouver

Chaudhari S, Aggarwal P, Murahari V, Rajpurohit T, Kalyan A, Narasimhan K, et al. RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs. ACM Computing Surveys. 2026;58(2):1-37. doi:10.1145/3743127

BibTeX
@article{Chaudhari2026,
  author = {Chaudhari, Shreyas and Aggarwal, Pranjal and Murahari, Vishvak and Rajpurohit, Tanmay and Kalyan, Ashwin and Narasimhan, Karthik and Deshpande, Ameet and Silva, Bruno Castro da},
  title = {RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs},
  journal = {ACM Computing Surveys},
  year = {2026},
  volume = {58},
  number = {2},
  pages = {1--37},
  doi = {10.1145/3743127}
}
Kostenlos starten →