EUVIMEDEUVIMED entdecken
Crossref

Deep Linear Networks for Matrix Completion—an Infinite Depth Limit

Nadav Cohen, Govind Menon, Zsolt Veraszto

SIAM Journal on Applied Dynamical Systems · 31.12.2023 · 22 (4) · 3208-3232

DOI: 10.1137/22m1530653 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Applied Dynamical Systems, online erschienen am 28.11.2023. Verfasst von Nadav Cohen und 2 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 21 Literaturangaben. Gefördert von: Israel Science Foundation, Division of Mathematical Sciences.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The deep linear network (DLN) is a model for implicit regularization in gradient based optimization of overparametrized learning architectures. Training the DLN corresponds to a Riemannian gradient flow, where the Riemannian metric is defined by the architecture of the network and the loss function is defined by the learning task. We extend this geometric framework, obtaining explicit expressions for the volume form, including the case when the network has infinite depth. We investigate the link between the Riemannian geometry and the training asymptotics for matrix completion with rigorous analysis and numerics. We propose that under small initialization, implicit regularization is a result of bias towards high state space volume.

Förderung

  • Israel Science Foundation
  • Division of Mathematical Sciences

Zitieren

APA

Cohen, N., Menon, G., & Veraszto, Z. (2023). Deep Linear Networks for Matrix Completion—an Infinite Depth Limit. SIAM Journal on Applied Dynamical Systems, 22(4), 3208-3232. https://doi.org/10.1137/22m1530653

Vancouver

Cohen N, Menon G, Veraszto Z. Deep Linear Networks for Matrix Completion—an Infinite Depth Limit. SIAM Journal on Applied Dynamical Systems. 2023;22(4):3208-3232. doi:10.1137/22m1530653

BibTeX
@article{Cohen2023,
  author = {Cohen, Nadav and Menon, Govind and Veraszto, Zsolt},
  title = {Deep Linear Networks for Matrix Completion—an Infinite Depth Limit},
  journal = {SIAM Journal on Applied Dynamical Systems},
  year = {2023},
  volume = {22},
  number = {4},
  pages = {3208--3232},
  doi = {10.1137/22m1530653}
}
Kostenlos starten →