Sparse-Coding Variational Autoencoders
Neural Computation · 19.11.2024 · 36 (12) · 2571-2601
DOI: 10.1162/neco_a_01715 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neural Computation, online erschienen am 19.11.2024. Verfasst von Victor Geadah und 4 weiteren Autor:innen. Beteiligte Einrichtung: Program in Applied and Computational Mathematics, Princeton University, Princeton, NJ 08544, U.S.A. victor.geadah@princeton.edu. Hauptthemen laut MEDLINE: Neural Networks, Computer. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 89 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NEI NIH HHS, NINDS NIH HHS, NIMH NIH HHS.
Abstract (Originalsprache)
The sparse coding model posits that the visual system has evolved to efficiently code natural stimuli using a sparse set of features from an overcomplete dictionary. The original sparse coding model suffered from two key limitations; however: (1) computing the neural response to an image patch required minimizing a nonlinear objective function via recurrent dynamics and (2) fitting relied on approximate inference methods that ignored uncertainty. Although subsequent work has developed several methods to overcome these obstacles, we propose a novel solution inspired by the variational autoencoder (VAE) framework. We introduce the sparse coding variational autoencoder (SVAE), which augments the sparse coding model with a probabilistic recognition model parameterized by a deep neural network. This recognition model provides a neurally plausible feedforward implementation for the mapping from image patches to neural activities and enables a principled method for fitting the sparse coding model to data via maximization of the evidence lower bound (ELBO). The SVAE differs from standard VAEs in three key respects: the latent representation is overcomplete (there are more latent dimensions than image pixels), the prior is sparse or heavy-tailed instead of gaussian, and the decoder network is a linear projection instead of a deep network. We fit the SVAE to natural image data under different assumed prior distributions and show that it obtains higher test performance than previous fitting methods. Finally, we examine the response properties of the recognition network and show that it captures important nonlinear properties of neurons in the early visual pathway.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Program in Applied and Computational Mathematics, Princeton University, Princeton, NJ 08544, U.S.A. victor.geadah@princeton.edu.
- Institute of Neuroscience, University of Oregon, Eugene, OR 97403, U.S.A. gabrielbarello@gmail.com.
- Department of Computer Science, Princeton University, Princeton, NJ 08544, U.S.A. daniel@danielgreenidge.com.
- Department of Biomedical Engineering, Department Center for Imaging Science, and Department Kavli Neuroscience Discovery Institute, Baltimore, MD 21218, U.S.A. adamsc@jhu.edu.
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, U.S.A. pillow@princeton.edu.
Förderung
- NEI NIH HHS
- NINDS NIH HHS
- NIMH NIH HHS
- NIDA NIH HHS
Zitieren
APA
Geadah, V., Barello, G., Greenidge, D., Charles, A. S., & Pillow, J. W. (2024). Sparse-Coding Variational Autoencoders. Neural Computation, 36(12), 2571-2601. https://doi.org/10.1162/neco_a_01715
Vancouver
Geadah V, Barello G, Greenidge D, Charles AS, Pillow JW. Sparse-Coding Variational Autoencoders. Neural Computation. 2024;36(12):2571-2601. doi:10.1162/neco_a_01715
BibTeX
@article{Geadah2024,
author = {Geadah, Victor and Barello, Gabriel and Greenidge, Daniel and Charles, Adam S and Pillow, Jonathan W},
title = {Sparse-Coding Variational Autoencoders},
journal = {Neural Computation},
year = {2024},
volume = {36},
number = {12},
pages = {2571--2601},
doi = {10.1162/neco_a_01715}
}
EUVIMED