EUVIMEDEUVIMED entdecken
Crossref

Exponential Capacity of Dense Associative Memories

Carlo Lucibello, Marc Mézard

Physical Review Letters · 13.02.2024 · 132 (7)

DOI: 10.1103/physrevlett.132.077301 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Physical Review Letters, online erschienen am 13.02.2024. Verfasst von Carlo Lucibello und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computing Sciences, Bocconi University, Milano 20136, Italy and Bocconi Institute for Data Science and Analytics (BIDSA), Milano 20136,…. Die Arbeit wurde bisher 43-mal zitiert (Crossref) und nennt 23 Literaturangaben. Gefördert von: European Commission, Ministero dell’Istruzione, dell’Università e della Ricerca, NextGeneration EU program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent generalizations of the Hopfield model of associative memories are able to store a number P of random patterns that grows exponentially with the number N of neurons, P=exp(αN). Besides the huge storage capacity, another interesting feature of these networks is their connection to the attention mechanism which is part of the Transformer architecture widely applied in deep learning. In this work, we study a generic family of pattern ensembles using a statistical mechanics analysis which gives exact asymptotic thresholds for the retrieval of a typical pattern, α_{1}, and lower bounds for the maximum of the load α for which all patterns can be retrieved, α_{c}, as well as sizes of attraction basins. We discuss in detail the cases of Gaussian and spherical patterns, and show that they display rich and qualitatively different phase diagrams.

Autor:innen und Institutionen

  • Department of Computing Sciences, Bocconi University, Milano 20136, Italy and Bocconi Institute for Data Science and Analytics (BIDSA), Milano 20136, Italy.

Förderung

  • European Commission
  • Ministero dell’Istruzione, dell’Università e della Ricerca
  • NextGeneration EU program

Zitieren

APA

Lucibello, C., & Mézard, M. (2024). Exponential Capacity of Dense Associative Memories. Physical Review Letters, 132(7). https://doi.org/10.1103/physrevlett.132.077301

Vancouver

Lucibello C, Mézard M. Exponential Capacity of Dense Associative Memories. Physical Review Letters. 2024;132(7). doi:10.1103/physrevlett.132.077301

BibTeX
@article{Lucibello2024,
  author = {Lucibello, Carlo and Mézard, Marc},
  title = {Exponential Capacity of Dense Associative Memories},
  journal = {Physical Review Letters},
  year = {2024},
  volume = {132},
  number = {7},
  doi = {10.1103/physrevlett.132.077301}
}
Kostenlos starten →