EUVIMEDEUVIMED entdecken
Crossref

On Adaptive Stochastic Heavy Ball Momentum for Solving Linear Systems

Yun Zeng, Deren Han, Yansheng Su, Jiaxin Xie

SIAM Journal on Matrix Analysis and Applications · 30.09.2024 · 45 (3) · 1259-1286

DOI: 10.1137/23m1575883 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Matrix Analysis and Applications, online erschienen am 09.07.2024. Verfasst von Yun Zeng und 3 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 61 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The stochastic heavy ball momentum (SHBM) method has gained considerable popularity as a scalable approach for solving large-scale optimization problems. However, one limitation of this method is its reliance on prior knowledge of certain problem parameters, such as singular values of a matrix. In this paper, we propose an adaptive variant of the SHBM method for solving stochastic problems that are reformulated from linear systems using user-defined distributions. Our adaptive SHBM (ASHBM) method utilizes iterative information to update the parameters, addressing an open problem in the literature regarding the adaptive learning of momentum parameters. We prove that our method converges linearly in expectation, with a better convergence bound compared to the basic method. Notably, we demonstrate that the deterministic version of our ASHBM algorithm can be reformulated as a variant of the conjugate gradient (CG) method, inheriting many of its appealing properties, such as finite-time convergence. Consequently, the ASHBM method can be further generalized to develop a brand-new framework of the stochastic CG method for solving linear systems. Our theoretical results are supported by numerical experiments.

Förderung

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities

Zitieren

APA

Zeng, Y., Han, D., Su, Y., & Xie, J. (2024). On Adaptive Stochastic Heavy Ball Momentum for Solving Linear Systems. SIAM Journal on Matrix Analysis and Applications, 45(3), 1259-1286. https://doi.org/10.1137/23m1575883

Vancouver

Zeng Y, Han D, Su Y, Xie J. On Adaptive Stochastic Heavy Ball Momentum for Solving Linear Systems. SIAM Journal on Matrix Analysis and Applications. 2024;45(3):1259-1286. doi:10.1137/23m1575883

BibTeX
@article{Zeng2024,
  author = {Zeng, Yun and Han, Deren and Su, Yansheng and Xie, Jiaxin},
  title = {On Adaptive Stochastic Heavy Ball Momentum for Solving Linear Systems},
  journal = {SIAM Journal on Matrix Analysis and Applications},
  year = {2024},
  volume = {45},
  number = {3},
  pages = {1259--1286},
  doi = {10.1137/23m1575883}
}
Kostenlos starten →