EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Stochastic gradient descent: a nonlinear time series perspective

Jiaqi Li, Zhipeng Lou, Stefan Richter, Wei Biao Wu

Journal of the Royal Statistical Society Series B: Statistical Methodology · 13.11.2026

DOI: 10.1093/jrsssb/qkag141 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the Royal Statistical Society Series B: Statistical Methodology, online erschienen am 13.11.2026. Verfasst von Jiaqi Li und 3 weiteren Autor:innen. Die Arbeit nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: NSF.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper investigates the statistical behaviour of stochastic gradient descent (SGD) with constant step size under the framework of iterated random functions. Compared with previous studies establishing the convergence of SGD in probability measure, e.g. Wasserstein distance, our approach directly provides the convergence in Euclidean distance by showing the geometric moment contraction of SGD. This new convergence can address the nonstationarity of SGD due to fixed initial points and can provide more refined asymptotics of SGD. Specifically, we prove quenched central limit theorem, invariance principle and α-stable convergence for averaged SGD (ASGD) regardless of initialization. Furthermore, we provide a novel perspective to understand the impact of step sizes in SGD by studying its derivative with respect to the step size. The existence of stationary solutions for the derivative processes is shown, enabling the use of interpolation to recover the full-trajectory of SGD. Subsequently, we utilize multiple step sizes and show an enhanced Richardson–Romberg extrapolation with improved bias representation, which brings ASGD estimates closer to the global optimum. Finally, we propose an online inference method and a bias-reduced variant for the extrapolated ASGD. Empirical confidence intervals are constructed and the coverage probabilities approach the nominal level as n grows.

Förderung

  • NSF

Zitieren

APA

Li, J., Lou, Z., Richter, S., & Wu, W. B. (2026). Stochastic gradient descent: a nonlinear time series perspective. Journal of the Royal Statistical Society Series B: Statistical Methodology. https://doi.org/10.1093/jrsssb/qkag141

Vancouver

Li J, Lou Z, Richter S, Wu WB. Stochastic gradient descent: a nonlinear time series perspective. Journal of the Royal Statistical Society Series B: Statistical Methodology. 2026. doi:10.1093/jrsssb/qkag141

BibTeX
@article{Li2026,
  author = {Li, Jiaqi and Lou, Zhipeng and Richter, Stefan and Wu, Wei Biao},
  title = {Stochastic gradient descent: a nonlinear time series perspective},
  journal = {Journal of the Royal Statistical Society Series B: Statistical Methodology},
  year = {2026},
  doi = {10.1093/jrsssb/qkag141}
}
Kostenlos starten →