EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Determinantal point process attention over grid cell code supports out of distribution generalization

Shanka Subhra Mondal iD, Steven Frankland, Taylor W Webb, Jonathan D Cohen iD

eLife · 2024

DOI: 10.7554/elife.89911 ↗

Kurz gefasst

Fachartikel in der Zeitschrift eLife, 2024. Verfasst von Shanka Subhra Mondal und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical and Computer Engineering, Princeton University, Princeton, United States. Hauptthemen laut MEDLINE: Neural Networks, Computer, Grid Cells. Die Arbeit nennt 78 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Office of Naval Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep neural networks have made tremendous gains in emulating human-like intelligence, and have been used increasingly as ways of understanding how the brain may solve the complex computational problems on which this relies. However, these still fall short of, and therefore fail to provide insight into how the brain supports strong forms of generalization of which humans are capable. One such case is out-of-distribution (OOD) generalization – successful performance on test examples that lie outside the distribution of the training set. Here, we identify properties of processing in the brain that may contribute to this ability. We describe a two-part algorithm that draws on specific features of neural computation to achieve OOD generalization, and provide a proof of concept by evaluating performance on two challenging cognitive tasks. First we draw on the fact that the mammalian brain represents metric spaces using grid cell code (e.g., in the entorhinal cortex): abstract representations of relational structure, organized in recurring motifs that cover the representational space. Second, we propose an attentional mechanism that operates over the grid cell code using determinantal point process (DPP), that we call DPP attention (DPP-A) – a transformation that ensures maximum sparseness in the coverage of that space. We show that a loss function that combines standard task-optimized error with DPP-A can exploit the recurring motifs in the grid cell code, and can be integrated with common architectures to achieve strong OOD generalization performance on analogy and arithmetic tasks. This provides both an interpretation of how the grid cell code in the mammalian brain may contribute to generalization performance, and at the same time a potential means for improving such capabilities in artificial neural networks.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Neural Networks, ComputerGrid CellsHumansAlgorithmsModels, NeurologicalAnimalsAttentionBrainEntorhinal Cortex

Stichwörter der Autor:innen

attentiondeterminantal point processgeneralizationgrid codeneurosciencenone

Autor:innen und Institutionen

  • Department of Electrical and Computer Engineering, Princeton University, Princeton, United States.
  • Princeton Neuroscience Institute, Princeton University, Princeton, United States.
  • Department of Psychology, University of California, Los Angeles, Los Angeles, United States.

ORCID-Profile: Shanka Subhra Mondal, Jonathan D Cohen

Förderung

  • Office of Naval Research

Zitieren

APA

Mondal, S. S., Frankland, S., Webb, T. W., & Cohen, J. D. (2024). Determinantal point process attention over grid cell code supports out of distribution generalization. eLife. https://doi.org/10.7554/elife.89911

Vancouver

Mondal SS, Frankland S, Webb TW, Cohen JD. Determinantal point process attention over grid cell code supports out of distribution generalization. eLife. 2024. doi:10.7554/elife.89911

BibTeX
@article{Mondal2024,
  author = {Mondal, Shanka Subhra and Frankland, Steven and Webb, Taylor W and Cohen, Jonathan D},
  title = {Determinantal point process attention over grid cell code supports out of distribution generalization},
  journal = {eLife},
  year = {2024},
  doi = {10.7554/elife.89911}
}
Kostenlos starten →