Stable Gradient-Adjusted Root Mean Square Propagation on Least Squares Problem
SIAM Journal on Mathematics of Data Science · 30.06.2026 · 8 (2) · 440-470
DOI: 10.1137/24m171262x ↗Kurz gefasst
Fachartikel in der Zeitschrift SIAM Journal on Mathematics of Data Science, online erschienen am 15.06.2026. Verfasst von Runze Li und 2 weiteren Autor:innen. Die Arbeit nennt 42 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Hong Kong Polytechnic University, Huawei Technologies.
Abstract (Originalsprache)
Root mean square propagation (RMSProp) is a first-order stochastic algorithm used widely in machine learning. In this paper, a stable gradient-adjusted RMSProp (SGA-RMSProp) with a minibatch stochastic gradient is proposed, and its properties are studied on the linear least squares problem (LLSP). R-linear convergence of the algorithm is established on the consistent LLSP. The algorithm is also proved to converge R-linearly to a neighborhood of the minimizer for the inconsistent case, with the region of the neighborhood being controlled by the batch size. Furthermore, numerical experiments are conducted to compare the performances of SGA-RMSProp, stochastic gradient descent (SGD), and the original RMSProp with different batch sizes. The faster initial convergence rate of SGA-RMSProp is observed through numerical experiments, and an adaptive strategy for switching from SGA-RMSProp to SGD is proposed, which combines the benefits of these two algorithms.
Förderung
- National Natural Science Foundation of China
- Hong Kong Polytechnic University
- Huawei Technologies
Zitieren
APA
Li, R., Xu, J., & Xing, W. (2026). Stable Gradient-Adjusted Root Mean Square Propagation on Least Squares Problem. SIAM Journal on Mathematics of Data Science, 8(2), 440-470. https://doi.org/10.1137/24m171262x
Vancouver
Li R, Xu J, Xing W. Stable Gradient-Adjusted Root Mean Square Propagation on Least Squares Problem. SIAM Journal on Mathematics of Data Science. 2026;8(2):440-470. doi:10.1137/24m171262x
BibTeX
@article{Li2026,
author = {Li, Runze and Xu, Jintao and Xing, Wenxun},
title = {Stable Gradient-Adjusted Root Mean Square Propagation on Least Squares Problem},
journal = {SIAM Journal on Mathematics of Data Science},
year = {2026},
volume = {8},
number = {2},
pages = {440--470},
doi = {10.1137/24m171262x}
}
EUVIMED