In Search of Dispersed Memories: Generative Diffusion Models Are Associative Memory Networks
Entropy · 29.04.2024 · 26 (5) · 381
DOI: 10.3390/e26050381 ↗Kurz gefasst
Fachartikel in der Zeitschrift Entropy, online erschienen am 29.04.2024. Verfasst von Luca Ambrogioni. Beteiligte Einrichtung: Donders Institute for Brain, Cognition, and Behaviour, Radboud University, 6525 XZ Nijmegen, The Netherlands. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Uncovering the mechanisms behind long-term memory is one of the most fascinating open problems in neuroscience and artificial intelligence. Artificial associative memory networks have been used to formalize important aspects of biological memory. Generative diffusion models are a type of generative machine learning techniques that have shown great performance in many tasks. Similar to associative memory systems, these networks define a dynamical system that converges to a set of target states. In this work, we show that generative diffusion models can be interpreted as energy-based models and that, when trained on discrete patterns, their energy function is (asymptotically) identical to that of modern Hopfield networks. This equivalence allows us to interpret the supervised training of diffusion models as a synaptic learning process that encodes the associative dynamics of a modern Hopfield network in the weight structure of a deep neural network. Leveraging this connection, we formulate a generalized framework for understanding the formation of long-term memory, where creative generation and memory recall can be seen as parts of a unified continuum.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Donders Institute for Brain, Cognition, and Behaviour, Radboud University, 6525 XZ Nijmegen, The Netherlands.
Zitieren
APA
Ambrogioni, L. (2024). In Search of Dispersed Memories: Generative Diffusion Models Are Associative Memory Networks. Entropy, 26(5), 381. https://doi.org/10.3390/e26050381
Vancouver
Ambrogioni L. In Search of Dispersed Memories: Generative Diffusion Models Are Associative Memory Networks. Entropy. 2024;26(5):381. doi:10.3390/e26050381
BibTeX
@article{Ambrogioni2024,
author = {Ambrogioni, Luca},
title = {In Search of Dispersed Memories: Generative Diffusion Models Are Associative Memory Networks},
journal = {Entropy},
year = {2024},
volume = {26},
number = {5},
pages = {381},
doi = {10.3390/e26050381}
}
EUVIMED