Individual differences in tail risk sensitive exploration using Bayes-adaptive Markov decision processes
eLife · 01.12.2025 · 13
DOI: 10.7554/elife.100366.3 ↗Kurz gefasst
Fachartikel in der Zeitschrift eLife, online erschienen am 01.12.2025. Verfasst von Tingke Shen und 1 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 72 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Max Planck Society, Alexander von Humboldt Foundation, Deutsche Forschungsgemeinschaft.
Abstract (Originalsprache)
Novelty is a double-edged sword for agents and animals alike: they might benefit from untapped resources or face unexpected costs or dangers such as predation. The conventional exploration/exploitation tradeoff is thus colored by risk sensitivity. A wealth of experiments has shown how animals solve this dilemma, for example, using intermittent approach. However, there are large individual differences in the nature of approach, and modeling has yet to elucidate how this might be based on animals’ differing prior expectations about reward and threat, and differing degrees of risk aversion. To capture these factors, we built a Bayes-adaptive Markov decision process model with three key components: an adaptive hazard function capturing potential predation, an intrinsic reward function providing the urge to explore, and a conditional value at risk (CVaR) objective, which is a contemporary measure of trait risk sensitivity. We fit this model to a coarse-grain abstraction of the behavior of 26 animals who freely explored a novel object in an open-field arena. We show that the model captures both quantitative (frequency, duration of exploratory bouts) and qualitative (with distinguished, cautious, tail-behind approach) features of behavior, including the substantial idiosyncrasies that were observed. Some animals begin with cautious exploration and quickly transition to a confident approach to maximize exploration for reward; we classify them as potentially more risk neutral and enjoying a flexible hazard prior. By contrast, other animals only ever approach in a cautious manner and display a form of self-censoring; they are characterized by potential risk aversion and high and inflexible hazard priors. Explaining risk-sensitive exploration using factorized parameters of reinforcement learning models could aid in the understanding, diagnosis, and treatment of psychiatric abnormalities such as anxiety disorders.
Förderung
- Max Planck Society
- Alexander von Humboldt Foundation
- Deutsche Forschungsgemeinschaft
- Else Kröner-Fresenius-Stiftung
Zitieren
APA
Shen, T., & Dayan, P. (2025). Individual differences in tail risk sensitive exploration using Bayes-adaptive Markov decision processes. eLife, 13. https://doi.org/10.7554/elife.100366.3
Vancouver
Shen T, Dayan P. Individual differences in tail risk sensitive exploration using Bayes-adaptive Markov decision processes. eLife. 2025;13. doi:10.7554/elife.100366.3
BibTeX
@article{Shen2025,
author = {Shen, Tingke and Dayan, Peter},
title = {Individual differences in tail risk sensitive exploration using Bayes-adaptive Markov decision processes},
journal = {eLife},
year = {2025},
volume = {13},
doi = {10.7554/elife.100366.3}
}
EUVIMED