EUVIMEDEUVIMED entdecken
Crossref

An Augmented Backward-Corrected Projector Splitting Integrator for Dynamical Low-Rank Training

Jonas Kusch, Steffen Schotthöfer, Alexandra Walter

SIAM Journal on Mathematics of Data Science · 30.09.2026 · 8 (3) · 820-849

DOI: 10.1137/25m1730673 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Mathematics of Data Science, online erschienen am 07.08.2026. Verfasst von Jonas Kusch und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: Helmholtz Information & Data Science School for Health, Helmholtz Information and Data Science Academy, U.S. Department of Energy.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Layer factorization has emerged as a widely used technique for training memory-efficient neural networks. However, layer factorization methods face several challenges, particularly a lack of robustness during the training process. To overcome this limitation, dynamical low-rank training methods have been developed, utilizing robust time integration techniques for low-rank matrix differential equations. Although these approaches facilitate efficient training, they still depend on computationally intensive QR and singular value decompositions of matrices with small rank. In this work, we introduce a novel low-rank training method that reduces the number of required QR decompositions. Our approach integrates an augmentation step into a projector-splitting scheme, ensuring that a subsequence of the projected gradient converges to zero in probability. We provide a rigorous theoretical analysis of the proposed method and demonstrate its effectiveness across multiple benchmarks.

Förderung

  • Helmholtz Information & Data Science School for Health
  • Helmholtz Information and Data Science Academy
  • U.S. Department of Energy

Zitieren

APA

Kusch, J., Schotthöfer, S., & Walter, A. (2026). An Augmented Backward-Corrected Projector Splitting Integrator for Dynamical Low-Rank Training. SIAM Journal on Mathematics of Data Science, 8(3), 820-849. https://doi.org/10.1137/25m1730673

Vancouver

Kusch J, Schotthöfer S, Walter A. An Augmented Backward-Corrected Projector Splitting Integrator for Dynamical Low-Rank Training. SIAM Journal on Mathematics of Data Science. 2026;8(3):820-849. doi:10.1137/25m1730673

BibTeX
@article{Kusch2026,
  author = {Kusch, Jonas and Schotthöfer, Steffen and Walter, Alexandra},
  title = {An Augmented Backward-Corrected Projector Splitting Integrator for Dynamical Low-Rank Training},
  journal = {SIAM Journal on Mathematics of Data Science},
  year = {2026},
  volume = {8},
  number = {3},
  pages = {820--849},
  doi = {10.1137/25m1730673}
}
Kostenlos starten →