Nonmonotonic Cyclic Stochastic Gradient Method
Bulletin of Chinese Applied Mathematics · 2025
DOI: 10.48014/bcam.20250527001 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bulletin of Chinese Applied Mathematics, 2025. Verfasst von Zhijie XIE und 1 weiteren Autor:innen.
Abstract (Originalsprache)
The Stochastic Gradient Descent (SGD) method is a widely used optimization method for machine learning, where the selection of the step size is a crucial factor for the performance of SGD. This paper combines the nonmonotonic line search technique with a cyclic update strategy for step size, to propose a nonmonotonic cyclic stochastic gradient method. For strongly convex, convex, and non-convex cases, we provide convergence analyses of the proposed algorithm. In numerical experiments, the proposed algorithm shows better performances compared to existing algorithms and is stable in terms of changes in hyperparameters.
Zitieren
APA
XIE, Z., & SUN, C. (2025). Nonmonotonic Cyclic Stochastic Gradient Method. Bulletin of Chinese Applied Mathematics. https://doi.org/10.48014/bcam.20250527001
Vancouver
XIE Z, SUN C. Nonmonotonic Cyclic Stochastic Gradient Method. Bulletin of Chinese Applied Mathematics. 2025. doi:10.48014/bcam.20250527001
BibTeX
@article{XIE2025,
author = {XIE, Zhijie and SUN, Cong},
title = {Nonmonotonic Cyclic Stochastic Gradient Method},
journal = {Bulletin of Chinese Applied Mathematics},
year = {2025},
doi = {10.48014/bcam.20250527001}
}
EUVIMED