EUVIMEDEUVIMED entdecken
Crossref

A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization with Non-Isolated Local Minima

Taehee Ko, Xiantao Li

Journal of Machine Learning · 06.06.2023 · 2 (2) · 138-160

DOI: 10.4208/jml.v2.n2.p138 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Machine Learning, online erschienen am 06.06.2023. Verfasst von Taehee Ko und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 41 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Loss functions with non-isolated minima have emerged in several machine-learning problems, creating a gap between theoretical predictions and observations in practice. In this paper, we formulate a new type of local convexity condition that is suitable to describe the behavior of loss functions near non-isolated minima. We show that such a condition is general enough to encompass many existing conditions. In addition, we study the local convergence of the stochastic gradient descent (SGD) method under this mild condition by adopting the notion of stochastic stability. In the convergence analysis, we establish concentration inequalities for the iterates in SGD, which can be used to interpret the empirical observation from some practical training results.

Zitieren

APA

Ko, T., & Li, X. (2023). A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization with Non-Isolated Local Minima. Journal of Machine Learning, 2(2), 138-160. https://doi.org/10.4208/jml.v2.n2.p138

Vancouver

Ko T, Li X. A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization with Non-Isolated Local Minima. Journal of Machine Learning. 2023;2(2):138-160. doi:10.4208/jml.v2.n2.p138

BibTeX
@article{Ko2023,
  author = {Ko, Taehee and Li, Xiantao},
  title = {A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization with Non-Isolated Local Minima},
  journal = {Journal of Machine Learning},
  year = {2023},
  volume = {2},
  number = {2},
  pages = {138--160},
  doi = {10.4208/jml.v2.n2.p138}
}
Kostenlos starten →