Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain
PLOS Computational Biology · 2024
DOI: 10.1371/journal.pcbi.1012058 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Computational Biology, 2024. Verfasst von Thirza Dado und 8 weiteren Autor:innen. Beteiligte Einrichtung: Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands. Hauptthemen laut MEDLINE: Visual Perception, Visual Cortex, Models, Neurological. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Dutch Research Council, Nederlandse Organisatie voor Wetenschappelijk Onderzoek.
Abstract (Originalsprache)
A challenging goal of neural coding is to characterize the neural representations underlying visual perception. To this end, multi-unit activity (MUA) of macaque visual cortex was recorded in a passive fixation task upon presentation of faces and natural images. We analyzed the relationship between MUA and latent representations of state-of-the-art deep generative models, including the conventional and feature-disentangled representations of generative adversarial networks (GANs) (i.e., z - and w -latents of StyleGAN, respectively) and language-contrastive representations of latent diffusion networks (i.e., CLIP-latents of Stable Diffusion). A mass univariate neural encoding analysis of the latent representations showed that feature-disentangled w representations outperform both z and CLIP representations in explaining neural responses. Further, w -latent features were found to be positioned at the higher end of the complexity gradient which indicates that they capture visual information relevant to high-level neural activity. Subsequently, a multivariate neural decoding analysis of the feature-disentangled representations resulted in state-of-the-art spatiotemporal reconstructions of visual perception. Taken together, our results not only highlight the important role of feature-disentanglement in shaping high-level neural representations underlying visual perception but also serve as an important benchmark for the future of neural coding.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands.
- Department of Vision and Cognition, Netherlands Institute for Neuroscience, Amsterdam, Netherlands.
- Laboratory of Visual Brain Therapy, Sorbonne University, Paris, France.
- Department of Integrative Neurophysiology, VU Amsterdam, Amsterdam, Netherlands.
- Department of Psychiatry, Amsterdam UMC, Amsterdam, Netherlands.
ORCID-Profile: Thirza Dado, Pieter Roelfsema
Förderung
- Dutch Research Council
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
Zitieren
APA
Dado, T., Papale, P., Lozano, A., Le, L., Wang, F., van Gerven, M., Roelfsema, P., Güçlütürk, Y., & Güçlü, U. (2024). Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1012058
Vancouver
Dado T, Papale P, Lozano A, Le L, Wang F, van Gerven M, et al. Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain. PLOS Computational Biology. 2024. doi:10.1371/journal.pcbi.1012058
BibTeX
@article{Dado2024,
author = {Dado, Thirza and Papale, Paolo and Lozano, Antonio and Le, Lynn and Wang, Feng and van Gerven, Marcel and Roelfsema, Pieter and Güçlütürk, Yağmur and Güçlü, Umut},
title = {Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain},
journal = {PLOS Computational Biology},
year = {2024},
doi = {10.1371/journal.pcbi.1012058}
}
EUVIMED