EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias

Itay Itzhak, Gabriel Stanovsky, Nir Rosenfeld, Yonatan Belinkov

Transactions of the Association for Computational Linguistics · 04.06.2024 · 12 · 771-785

DOI: 10.1162/tacl_a_00673 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, erschienen am 04.06.2024. Verfasst von Itay Itzhak und 3 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent studies show that instruction tuning (IT) and reinforcement learning from human feedback (RLHF) improve the abilities of large language models (LMs) dramatically. While these tuning methods can help align models with human objectives and generate high-quality text, not much is known about their potential adverse effects. In this work, we investigate the effect of IT and RLHF on decision making and reasoning in LMs, focusing on three cognitive biases—the decoy effect, the certainty effect, and the belief bias—all of which are known to influence human decision-making and reasoning. Our findings highlight the presence of these biases in various models from the GPT-3, Mistral, and T5 families. Notably, we find a stronger presence of biases in models that have undergone instruction tuning, such as Flan-T5, Mistral-Instruct, GPT3.5, and GPT4. Our work constitutes a step toward comprehending cognitive biases in instruction-tuned LMs, which is crucial for the development of more reliable and unbiased language models.1

Zitieren

APA

Itzhak, I., Stanovsky, G., Rosenfeld, N., & Belinkov, Y. (2024). Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias. Transactions of the Association for Computational Linguistics, 12, 771-785. https://doi.org/10.1162/tacl_a_00673

Vancouver

Itzhak I, Stanovsky G, Rosenfeld N, Belinkov Y. Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias. Transactions of the Association for Computational Linguistics. 2024;12:771-785. doi:10.1162/tacl_a_00673

BibTeX
@article{Itzhak2024,
  author = {Itzhak, Itay and Stanovsky, Gabriel and Rosenfeld, Nir and Belinkov, Yonatan},
  title = {Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  volume = {12},
  pages = {771--785},
  doi = {10.1162/tacl_a_00673}
}
Kostenlos starten →