EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

New efficient accelerated and stochastic gradient descent algorithms based on locally Lipschitz gradient constants

Nguyen Phung Hai Chung, Pham Thi Hoai, Hoang Van Chung

Journal of Convex Analysis · 2027 · 34 (1) · 271-293

DOI: 10.68381/jca34020 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Convex Analysis, 2027. Verfasst von Nguyen Phung Hai Chung und 2 weiteren Autor:innen. Die Arbeit nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper revisits the recently introduced stepsize in the gradient descent scheme called NGD, proposed by Hoai et al. We first investigate the NGD stepsize in conjunction with two well-known acceleration techniques: the Heavy Ball method and Nesterov's method. In the convex setting of unconstrained nonlinear optimization problems, we establish the ergodic convergence of the iterates generated by the accelerated versions of NGD at a sublinear rate. We introduce stochastic versions of the proposed accelerated algorithms and analyze their convergence in nonconvex settings. Although our proposed algorithms require global Lipschitz continuity of the gradient, we do not utilize the global Lipschitz constant during computations. Instead, we leverage information about local Lipschitz constants derived from previous iterations. Numerical experiments on various practical machine learning and deep learning problems demonstrate the efficiency of our proposed methods compared to the recent ones.

Zitieren

APA

Chung, N. P. H., Hoai, P. T., & Chung, H. V. (2027). New efficient accelerated and stochastic gradient descent algorithms based on locally Lipschitz gradient constants. Journal of Convex Analysis, 34(1), 271-293. https://doi.org/10.68381/jca34020

Vancouver

Chung NPH, Hoai PT, Chung HV. New efficient accelerated and stochastic gradient descent algorithms based on locally Lipschitz gradient constants. Journal of Convex Analysis. 2027;34(1):271-293. doi:10.68381/jca34020

BibTeX
@article{Chung2027,
  author = {Chung, Nguyen Phung Hai and Hoai, Pham Thi and Chung, Hoang Van},
  title = {New efficient accelerated and stochastic gradient descent algorithms based on locally Lipschitz gradient constants},
  journal = {Journal of Convex Analysis},
  year = {2027},
  volume = {34},
  number = {1},
  pages = {271--293},
  doi = {10.68381/jca34020}
}
Kostenlos starten →