EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain

Thirza Dado iD, Paolo Papale, Antonio Lozano, Lynn Le, Feng Wang, Marcel van Gerven, Pieter Roelfsema iD, Yağmur Güçlütürk, Umut Güçlü

PLOS Computational Biology · 2024

DOI: 10.1371/journal.pcbi.1012058 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, 2024. Verfasst von Thirza Dado und 8 weiteren Autor:innen. Beteiligte Einrichtung: Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands. Hauptthemen laut MEDLINE: Visual Perception, Visual Cortex, Models, Neurological. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Dutch Research Council, Nederlandse Organisatie voor Wetenschappelijk Onderzoek.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A challenging goal of neural coding is to characterize the neural representations underlying visual perception. To this end, multi-unit activity (MUA) of macaque visual cortex was recorded in a passive fixation task upon presentation of faces and natural images. We analyzed the relationship between MUA and latent representations of state-of-the-art deep generative models, including the conventional and feature-disentangled representations of generative adversarial networks (GANs) (i.e., z - and w -latents of StyleGAN, respectively) and language-contrastive representations of latent diffusion networks (i.e., CLIP-latents of Stable Diffusion). A mass univariate neural encoding analysis of the latent representations showed that feature-disentangled w representations outperform both z and CLIP representations in explaining neural responses. Further, w -latent features were found to be positioned at the higher end of the complexity gradient which indicates that they capture visual information relevant to high-level neural activity. Subsequently, a multivariate neural decoding analysis of the feature-disentangled representations resulted in state-of-the-art spatiotemporal reconstructions of visual perception. Taken together, our results not only highlight the important role of feature-disentanglement in shaping high-level neural representations underlying visual perception but also serve as an important benchmark for the future of neural coding.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Visual PerceptionVisual CortexModels, NeurologicalAnimalsMacaca mulattaComputational BiologyNeural Networks, ComputerPhotic StimulationMaleNeuronsBrain

Autor:innen und Institutionen

  • Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands.
  • Department of Vision and Cognition, Netherlands Institute for Neuroscience, Amsterdam, Netherlands.
  • Laboratory of Visual Brain Therapy, Sorbonne University, Paris, France.
  • Department of Integrative Neurophysiology, VU Amsterdam, Amsterdam, Netherlands.
  • Department of Psychiatry, Amsterdam UMC, Amsterdam, Netherlands.

ORCID-Profile: Thirza Dado, Pieter Roelfsema

Förderung

  • Dutch Research Council
  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek

Zitieren

APA

Dado, T., Papale, P., Lozano, A., Le, L., Wang, F., van Gerven, M., Roelfsema, P., Güçlütürk, Y., & Güçlü, U. (2024). Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1012058

Vancouver

Dado T, Papale P, Lozano A, Le L, Wang F, van Gerven M, et al. Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain. PLOS Computational Biology. 2024. doi:10.1371/journal.pcbi.1012058

BibTeX
@article{Dado2024,
  author = {Dado, Thirza and Papale, Paolo and Lozano, Antonio and Le, Lynn and Wang, Feng and van Gerven, Marcel and Roelfsema, Pieter and Güçlütürk, Yağmur and Güçlü, Umut},
  title = {Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain},
  journal = {PLOS Computational Biology},
  year = {2024},
  doi = {10.1371/journal.pcbi.1012058}
}
Kostenlos starten →