EUVIMEDEUVIMED entdecken
Crossref

Mechanism for feature learning in neural networks and backpropagation-free machine learning models

Adityanarayanan Radhakrishnan iD, Daniel Beaglehole iD, Parthe Pandit iD, Mikhail Belkin iD

Science · 29.03.2024 · 383 (6690) · 1461-1467

DOI: 10.1126/science.adi5639 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Science, erschienen am 29.03.2024. Verfasst von Adityanarayanan Radhakrishnan und 3 weiteren Autor:innen. Beteiligte Einrichtung: Harvard School of Engineering and Applied Sciences, Cambridge, MA 02138, USA. Die Arbeit wurde bisher 81-mal zitiert (Crossref) und nennt 72 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Understanding how neural networks learn features, or relevant patterns in data, for prediction is necessary for their reliable use in technological and scientific applications. In this work, we presented a unifying mathematical mechanism, known as average gradient outer product (AGOP), that characterized feature learning in neural networks. We provided empirical evidence that AGOP captured features learned by various neural network architectures, including transformer-based language models, convolutional networks, multilayer perceptrons, and recurrent neural networks. Moreover, we demonstrated that AGOP, which is backpropagation-free, enabled feature learning in machine learning models, such as kernel machines, that a priori could not identify task-specific features. Overall, we established a fundamental mechanism that captured feature learning in neural networks and enabled feature learning in general machine learning models.

Autor:innen und Institutionen

  • Harvard School of Engineering and Applied Sciences, Cambridge, MA 02138, USA.
  • Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA.
  • Computer Science and Engineering, UC San Diego, La Jolla, CA 92093, USA.
  • Center for Machine Intelligence and Data Science, IIT Bombay, Mumbai 400076, India.
  • Halıcıoğlu Data Science Institute, UC San Diego, La Jolla, CA 92093, USA.

ORCID-Profile: Adityanarayanan Radhakrishnan, Daniel Beaglehole, Parthe Pandit, Mikhail Belkin

Zitieren

APA

Radhakrishnan, A., Beaglehole, D., Pandit, P., & Belkin, M. (2024). Mechanism for feature learning in neural networks and backpropagation-free machine learning models. Science, 383(6690), 1461-1467. https://doi.org/10.1126/science.adi5639

Vancouver

Radhakrishnan A, Beaglehole D, Pandit P, Belkin M. Mechanism for feature learning in neural networks and backpropagation-free machine learning models. Science. 2024;383(6690):1461-1467. doi:10.1126/science.adi5639

BibTeX
@article{Radhakrishnan2024,
  author = {Radhakrishnan, Adityanarayanan and Beaglehole, Daniel and Pandit, Parthe and Belkin, Mikhail},
  title = {Mechanism for feature learning in neural networks and backpropagation-free machine learning models},
  journal = {Science},
  year = {2024},
  volume = {383},
  number = {6690},
  pages = {1461--1467},
  doi = {10.1126/science.adi5639}
}
Kostenlos starten →