EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Stable tensor neural networks for efficient deep learning

Elizabeth Newman, Lior Horesh, Haim Avron, Misha E Kilmer

Frontiers in Big Data · 30.05.2024 · 7

DOI: 10.3389/fdata.2024.1363978 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Big Data, online erschienen am 30.05.2024. Verfasst von Elizabeth Newman und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Mathematics, Emory University, Atlanta, GA, United States. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Learning from complex, multidimensional data has become central to computational mathematics, and among the most successful high-dimensional function approximators are deep neural networks (DNNs). Training DNNs is posed as an optimization problem to learn network weights or parameters that well-approximate a mapping from input to target data. Multiway data or tensors arise naturally in myriad ways in deep learning, in particular as input data and as high-dimensional weights and features extracted by the network, with the latter often being a bottleneck in terms of speed and memory. In this work, we leverage tensor representations and processing to efficiently parameterize DNNs when learning from high-dimensional data. We propose tensor neural networks (t-NNs), a natural extension of traditional fully-connected networks, that can be trained efficiently in a reduced, yet more powerful parameter space. Our t-NNs are built upon matrix-mimetic tensor-tensor products, which retain algebraic properties of matrix multiplication while capturing high-dimensional correlations. Mimeticity enables t-NNs to inherit desirable properties of modern DNN architectures. We exemplify this by extending recent work on stable neural networks, which interpret DNNs as discretizations of differential equations, to our multidimensional framework. We provide empirical evidence of the parametric advantages of t-NNs on dimensionality reduction using autoencoders and classification using fully-connected and stable variants on benchmark imaging datasets MNIST and CIFAR-10.

Schlagwörter

Stichwörter der Autor:innen

deep learningimage classificationinverse problemsmachine learningtensor algebra

Autor:innen und Institutionen

  • Department of Mathematics, Emory University, Atlanta, GA, United States.
  • Mathematics and Theoretical Computer Science, IBM TJ Watson Research Center, Yorktown, NY, United States.
  • Department of Applied Mathematics, Tel Aviv University, Tel Aviv-Yafo, Israel.
  • Department of Mathematics, Tufts University, Medford, MA, United States.

Zitieren

APA

Newman, E., Horesh, L., Avron, H., & Kilmer, M. E. (2024). Stable tensor neural networks for efficient deep learning. Frontiers in Big Data, 7. https://doi.org/10.3389/fdata.2024.1363978

Vancouver

Newman E, Horesh L, Avron H, Kilmer ME. Stable tensor neural networks for efficient deep learning. Frontiers in Big Data. 2024;7. doi:10.3389/fdata.2024.1363978

BibTeX
@article{Newman2024,
  author = {Newman, Elizabeth and Horesh, Lior and Avron, Haim and Kilmer, Misha E},
  title = {Stable tensor neural networks for efficient deep learning},
  journal = {Frontiers in Big Data},
  year = {2024},
  volume = {7},
  doi = {10.3389/fdata.2024.1363978}
}
Kostenlos starten →