Evolving choice hysteresis in reinforcement learning: Comparing the adaptive value of positivity bias and gradual perseveration
Proceedings of the National Academy of Sciences · 2025
DOI: 10.1073/pnas.2422144122 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Proceedings of the National Academy of Sciences, 2025. Verfasst von Isabelle Hoxha und 2 weiteren Autor:innen. Beteiligte Einrichtung: Département d'Etudes Cognitives, École Normale Supérieure, Université de Recherche Paris Sciences et Lettres, Paris 75005, France. Hauptthemen laut MEDLINE: Reinforcement, Psychology, Choice Behavior, Learning. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: EC | European Research Council (ERC), Agence Nationale de la Recherche (ANR), Alexander von Humboldt-Stiftung (AvH).
Abstract (Originalsprache)
The tendency to repeat past choices more often than expected from the history of outcomes has been repeatedly empirically observed in reinforcement learning experiments. It can be explained by at least two computational processes: asymmetric update and (gradual) choice perseveration. A recent meta-analysis showed that both mechanisms are detectable in human reinforcement learning. However, while their descriptive value seems to be well established, they have not been compared regarding their possible adaptive value. In this study, we address this gap by simulating reinforcement learning agents in a variety of environments with a variant of an evolutionary algorithm. Our results show that positivity bias (in the form of asymmetric update) is evolutionary stable in many situations, while the emergence of gradual perseveration is less systematic and robust. Overall, our results illustrate that biases can be adaptive and selected by evolution, in an environment-specific manner.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Département d'Etudes Cognitives, École Normale Supérieure, Université de Recherche Paris Sciences et Lettres, Paris 75005, France.
- Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et de la Recherche Médicale, Paris 75005, France.
ORCID-Profile: Isabelle Hoxha, Stefano Palminteri
Förderung
- EC | European Research Council (ERC)
- Agence Nationale de la Recherche (ANR)
- Alexander von Humboldt-Stiftung (AvH)
Zitieren
APA
Hoxha, I., Sperber, L., & Palminteri, S. (2025). Evolving choice hysteresis in reinforcement learning: Comparing the adaptive value of positivity bias and gradual perseveration. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2422144122
Vancouver
Hoxha I, Sperber L, Palminteri S. Evolving choice hysteresis in reinforcement learning: Comparing the adaptive value of positivity bias and gradual perseveration. Proceedings of the National Academy of Sciences. 2025. doi:10.1073/pnas.2422144122
BibTeX
@article{Hoxha2025,
author = {Hoxha, Isabelle and Sperber, Léo and Palminteri, Stefano},
title = {Evolving choice hysteresis in reinforcement learning: Comparing the adaptive value of positivity bias and gradual perseveration},
journal = {Proceedings of the National Academy of Sciences},
year = {2025},
doi = {10.1073/pnas.2422144122}
}
EUVIMED