EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Neural representations of the perception of handwritten digits and visual objects from a convolutional neural network compared to humans

Juhyeon Lee, Minyoung Jung, Niv Lustig, Jong-Hwan Lee iD

Human Brain Mapping · 2023

DOI: 10.1002/hbm.26189 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Human Brain Mapping, 2023. Verfasst von Juhyeon Lee und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea. Hauptthemen laut MEDLINE: Visual Perception, Brain. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 83 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Electronics and Telecommunications Research Institute, National Research Foundation of Korea.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We investigated neural representations for visual perception of 10 handwritten digits and six visual objects from a convolutional neural network (CNN) and humans using functional magnetic resonance imaging (fMRI). Once our CNN model was fine‐tuned using a pre‐trained VGG16 model to recognize the visual stimuli from the digit and object categories, representational similarity analysis (RSA) was conducted using neural activations from fMRI and feature representations from the CNN model across all 16 classes. The encoded neural representation of the CNN model exhibited the hierarchical topography mapping of the human visual system. The feature representations in the lower convolutional (Conv) layers showed greater similarity with the neural representations in the early visual areas and parietal cortices, including the posterior cingulate cortex. The feature representations in the higher Conv layers were encoded in the higher‐order visual areas, including the ventral/medial/dorsal stream and middle temporal complex. The neural representations in the classification layers were observed mainly in the ventral stream visual cortex (including the inferior temporal cortex), superior parietal cortex, and prefrontal cortex. There was a surprising similarity between the neural representations from the CNN model and the neural representations for human visual perception in the context of the perception of digits versus objects, particularly in the primary visual and associated areas. This study also illustrates the uniqueness of human visual perception. Unlike the CNN model, the neural representation of digits and objects for humans is more widely distributed across the whole brain, including the frontal and temporal areas.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Visual PerceptionBrainHumansMagnetic Resonance ImagingNeural Networks, ComputerParietal LobeBrain MappingPattern Recognition, Visual

Stichwörter der Autor:innen

convolutional neural networkfunctional magnetic resonance imaginghandwritten digitsrepresentational similarity analysisvisual objectsvisual perception

Autor:innen und Institutionen

  • Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea.

ORCID-Profile: Jong-Hwan Lee

Förderung

  • Electronics and Telecommunications Research Institute
  • National Research Foundation of Korea

Zitieren

APA

Lee, J., Jung, M., Lustig, N., & Lee, J. H. (2023). Neural representations of the perception of handwritten digits and visual objects from a convolutional neural network compared to humans. Human Brain Mapping. https://doi.org/10.1002/hbm.26189

Vancouver

Lee J, Jung M, Lustig N, Lee JH. Neural representations of the perception of handwritten digits and visual objects from a convolutional neural network compared to humans. Human Brain Mapping. 2023. doi:10.1002/hbm.26189

BibTeX
@article{Lee2023,
  author = {Lee, Juhyeon and Jung, Minyoung and Lustig, Niv and Lee, Jong-Hwan},
  title = {Neural representations of the perception of handwritten digits and visual objects from a convolutional neural network compared to humans},
  journal = {Human Brain Mapping},
  year = {2023},
  doi = {10.1002/hbm.26189}
}
Kostenlos starten →