Representation of locomotive action affordances in human behavior, brains, and deep neural networks
Proceedings of the National Academy of Sciences · 17.06.2025 · 122 (24)
DOI: 10.1073/pnas.2414005122 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the National Academy of Sciences, online erschienen am 12.06.2025. Verfasst von Clemens G Bartnik und 6 weiteren Autor:innen. Beteiligte Einrichtung: Informatics Institute, Video and Image Sense Lab, University of Amsterdam, Amsterdam 1098 XH, The Netherlands. Hauptthemen laut MEDLINE: Neural Networks, Computer, Visual Cortex, Brain, Locomotion. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO).
Abstract (Originalsprache)
To decide how to move around the world, we must determine which locomotive actions (e.g., walking, swimming, or climbing) are afforded by the immediate visual environment. The neural basis of our ability to recognize locomotive affordances is unknown. Here, we compare human behavioral annotations, functional MRI (fMRI) measurements, and deep neural network (DNN) activations to both indoor and outdoor real-world images to demonstrate that the human visual cortex represents locomotive action affordances in complex visual scenes. Hierarchical clustering of behavioral annotations of six possible locomotive actions show that humans group environments into distinct affordance clusters using at least three separate dimensions. Representational similarity analysis of multivoxel fMRI responses in the scene-selective visual cortex shows that perceived locomotive affordances are represented independently from other scene properties such as objects, surface materials, scene category, or global properties and independent of the task performed in the scanner. Visual feature activations from DNNs trained on object or scene classification as well as a range of other visual understanding tasks correlate comparatively lower with behavioral and neural representations of locomotive affordances than with object representations. Training DNNs directly on affordance labels or using affordance-centered language embeddings increases alignment with human behavior, but none of the tested models fully captures locomotive action affordance perception. These results uncover a type of representation in the human brain that reflects locomotive action affordances.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Informatics Institute, Video and Image Sense Lab, University of Amsterdam, Amsterdam 1098 XH, The Netherlands.
- Department of Brain and Cognition, Psychology Research Institute, University of Amsterdam, Amsterdam 1018 WS, The Netherlands.
ORCID-Profile: Clemens G Bartnik, Christina Sartzetaki, Abel Puigseslloses Sanchez, Elijah Molenkamp, Steven Bommer, Nikolina Vukšić, Iris I A Groen
Förderung
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO)
Zitieren
APA
Bartnik, C. G., Sartzetaki, C., Puigseslloses Sanchez, A., Molenkamp, E., Bommer, S., Vukšić, N., & Groen, I. I. A. (2025). Representation of locomotive action affordances in human behavior, brains, and deep neural networks. Proceedings of the National Academy of Sciences, 122(24). https://doi.org/10.1073/pnas.2414005122
Vancouver
Bartnik CG, Sartzetaki C, Puigseslloses Sanchez A, Molenkamp E, Bommer S, Vukšić N, et al. Representation of locomotive action affordances in human behavior, brains, and deep neural networks. Proceedings of the National Academy of Sciences. 2025;122(24). doi:10.1073/pnas.2414005122
BibTeX
@article{Bartnik2025,
author = {Bartnik, Clemens G and Sartzetaki, Christina and Puigseslloses Sanchez, Abel and Molenkamp, Elijah and Bommer, Steven and Vukšić, Nikolina and Groen, Iris I A},
title = {Representation of locomotive action affordances in human behavior, brains, and deep neural networks},
journal = {Proceedings of the National Academy of Sciences},
year = {2025},
volume = {122},
number = {24},
doi = {10.1073/pnas.2414005122}
}
EUVIMED