Neural network-based encoding in free-viewing fMRI with gaze-aware models
Neurons, Behavior, Data analysis, and Theory · 2026
DOI: 10.51628/001c.158956 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neurons, Behavior, Data analysis, and Theory, 2026. Verfasst von Dora Gozukara und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Representations learned by convolutional neural networks (CNNs) exhibit a remarkable resemblance to information processing patterns observed in the primate visual system on large neuroimaging datasets collected under diverse, naturalistic visual stimulation. However in most cases participants are instructed to maintain central fixation. This viewing condition diverges significantly from ecologically valid visual behaviour, suppresses activity in visually active regions, and imposes substantial cognitive load on the viewing task. We present a modification of the encoding model framework, adapting it for use with naturalistic vision datasets acquired under natural viewing conditions, without fixation, by incorporating eye-tracking data. Our gaze-aware encoding models were trained on the StudyForrest dataset, which features task-free naturalistic movie viewing. By combining eye-tracking data with the visual content of movie frames, we generate combined subject-wise gaze-stimulus specific feature time series. These time series are constructed by sampling only the locally and temporally relevant elements of the CNN feature map for each fixation. Our results demonstrate that gaze-aware encoding models match the performance of conventional encoding models with more than 100 times fewer model parameters. This parameter reduction makes encoding models more tractable for smaller datasets. Gaze-aware encoding models were especially beneficial for participants with more dynamic eye-movement patterns. Therefore, this approach opens the door to more ecologically valid models that can be built in more naturalistic settings, such as playing games or navigating virtual environments.
Zitieren
APA
Gozukara, D., Ahmad, N., Seeliger, K., Oetringer, D., & Geerligs, L. (2026). Neural network-based encoding in free-viewing fMRI with gaze-aware models. Neurons, Behavior, Data analysis, and Theory. https://doi.org/10.51628/001c.158956
Vancouver
Gozukara D, Ahmad N, Seeliger K, Oetringer D, Geerligs L. Neural network-based encoding in free-viewing fMRI with gaze-aware models. Neurons, Behavior, Data analysis, and Theory. 2026. doi:10.51628/001c.158956
BibTeX
@article{Gozukara2026,
author = {Gozukara, Dora and Ahmad, Nasir and Seeliger, Katja and Oetringer, Djamari and Geerligs, Linda},
title = {Neural network-based encoding in free-viewing fMRI with gaze-aware models},
journal = {Neurons, Behavior, Data analysis, and Theory},
year = {2026},
doi = {10.51628/001c.158956}
}
EUVIMED