EUVIMEDEUVIMED entdecken
Crossref

Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems

Levon Nurbekyan, Wanzhou Lei, Yunan Yang

SIAM Journal on Scientific Computing · 31.08.2023 · 45 (4) · A1621-A1655

DOI: 10.1137/22m1477805 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Scientific Computing, online erschienen am 10.07.2023. Verfasst von Levon Nurbekyan und 2 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 55 Literaturangaben. Gefördert von: AFOSR, Courant Institute of Mathematical Sciences, New York University, NSF.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We propose efficient numerical schemes for implementing the natural gradient descent (NGD) for a broad range of metric spaces with applications to PDE-based optimization problems. Our technique represents the natural gradient direction as a solution to a standard least-squares problem. Hence, instead of calculating, storing, or inverting the information matrix directly, we apply efficient methods from numerical linear algebra. We treat both scenarios where the Jacobian, i.e., the derivative of the state variable with respect to the parameter, is either explicitly known or implicitly given through constraints. We can thus reliably compute several natural NGDs for a large-scale parameter space. In particular, we are able to compute Wasserstein NGD in thousands of dimensions, which was believed to be out of reach. Finally, our numerical results shed light on the qualitative differences between the standard gradient descent and various NGD methods based on different metric spaces in nonconvex optimization problems.

Förderung

  • AFOSR
  • Courant Institute of Mathematical Sciences, New York University
  • NSF
  • Walter Haefner Foundation
  • ETH Zürich Foundation

Zitieren

APA

Nurbekyan, L., Lei, W., & Yang, Y. (2023). Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems. SIAM Journal on Scientific Computing, 45(4), A1621-A1655. https://doi.org/10.1137/22m1477805

Vancouver

Nurbekyan L, Lei W, Yang Y. Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems. SIAM Journal on Scientific Computing. 2023;45(4):A1621-A1655. doi:10.1137/22m1477805

BibTeX
@article{Nurbekyan2023,
  author = {Nurbekyan, Levon and Lei, Wanzhou and Yang, Yunan},
  title = {Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems},
  journal = {SIAM Journal on Scientific Computing},
  year = {2023},
  volume = {45},
  number = {4},
  pages = {A1621--A1655},
  doi = {10.1137/22m1477805}
}
Kostenlos starten →