EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Resource-Rational Account of Human Eye Movements During Immersive Visual Search

Angela Radulescu iD, Bas van Opheusden, Frederick Callaway iD, Thomas L Griffiths, James M Hillis

Open Mind · 01.02.2026 · 10 · 91-117

DOI: 10.1162/opmi.a.322 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Open Mind, online erschienen am 01.02.2026. Verfasst von Angela Radulescu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Facebook/Meta.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The nature of eye movements during visual search has been widely studied in cognitive science. Virtual reality (VR) paradigms are an opportunity to test whether computational models of search can predict naturalistic search behavior. However, existing ideal observer models are constrained by strong assumptions about the structure of the world, rendering them impractical for modeling the complexity of environments which can be studied in VR. To address these limitations, we modeled immersive visual search as a reinforcement learning problem, in which sequential decisions are made over a multidimensional representation of the environment learned by a convolutional neural network. In our formulation, RL agents learned a policy over latent states—effectively solving what is known as a meta–Markov decision process (meta-MDP), where each decision concerns how to allocate attention to information in the environment. Training deep-RL agents on the meta-MDP showed that learned (i.e., optimal) search policies converge to a classic ideal-observer model of search developed for simple (1D) stimuli. We compared the learned resource-rational policy with human gaze data from a visual-search experiment conducted in VR and found qualitative and quantitative alignment between model predictions and human behavior. However, both the model’s simulated performance and its correspondence with human behavior depended strongly on the representational features available to the policy. These results suggest that naturalistic visual search behavior can partially be explained by resource-rational allocation of limited cognitive resources, and the choice of representation influences the degree of alignment between model and human behavior.

Schlagwörter

Stichwörter der Autor:innen

deep reinforcement learningresource rationalityvirtual realityvisual search

Autor:innen und Institutionen

  • Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
  • OpenAI.
  • Department of Psychology, New York University, New York, NY, USA.
  • Departments of Psychology and Computer Science, Princeton University, Princeton, NJ, USA.
  • Meta.

ORCID-Profile: Angela Radulescu, Frederick Callaway

Förderung

  • Facebook/Meta

Zitieren

APA

Radulescu, A., van Opheusden, B., Callaway, F., Griffiths, T. L., & Hillis, J. M. (2026). A Resource-Rational Account of Human Eye Movements During Immersive Visual Search. Open Mind, 10, 91-117. https://doi.org/10.1162/opmi.a.322

Vancouver

Radulescu A, van Opheusden B, Callaway F, Griffiths TL, Hillis JM. A Resource-Rational Account of Human Eye Movements During Immersive Visual Search. Open Mind. 2026;10:91-117. doi:10.1162/opmi.a.322

BibTeX
@article{Radulescu2026,
  author = {Radulescu, Angela and van Opheusden, Bas and Callaway, Frederick and Griffiths, Thomas L and Hillis, James M},
  title = {A Resource-Rational Account of Human Eye Movements During Immersive Visual Search},
  journal = {Open Mind},
  year = {2026},
  volume = {10},
  pages = {91--117},
  doi = {10.1162/opmi.a.322}
}
Kostenlos starten →