EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Recent Advances in Stochastic Gradient Descent in Deep Learning

Yingjie Tian, Yuqi Zhang, Haibin Zhang

Mathematics · 29.01.2023 · 11 (3) · 682

DOI: 10.3390/math11030682 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematics, online erschienen am 29.01.2023. Verfasst von Yingjie Tian und 2 weiteren Autor:innen. Die Arbeit wurde bisher 281-mal zitiert (Crossref) und nennt 132 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the age of artificial intelligence, the best approach to handling huge amounts of data is a tremendously motivating and hard problem. Among machine learning models, stochastic gradient descent (SGD) is not only simple but also very effective. This study provides a detailed analysis of contemporary state-of-the-art deep learning applications, such as natural language processing (NLP), visual data processing, and voice and audio processing. Following that, this study introduces several versions of SGD and its variant, which are already in the PyTorch optimizer, including SGD, Adagrad, adadelta, RMSprop, Adam, AdamW, and so on. Finally, we propose theoretical conditions under which these methods are applicable and discover that there is still a gap between theoretical conditions under which the algorithms converge and practical applications, and how to bridge this gap is a question for the future.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Tian, Y., Zhang, Y., & Zhang, H. (2023). Recent Advances in Stochastic Gradient Descent in Deep Learning. Mathematics, 11(3), 682. https://doi.org/10.3390/math11030682

Vancouver

Tian Y, Zhang Y, Zhang H. Recent Advances in Stochastic Gradient Descent in Deep Learning. Mathematics. 2023;11(3):682. doi:10.3390/math11030682

BibTeX
@article{Tian2023,
  author = {Tian, Yingjie and Zhang, Yuqi and Zhang, Haibin},
  title = {Recent Advances in Stochastic Gradient Descent in Deep Learning},
  journal = {Mathematics},
  year = {2023},
  volume = {11},
  number = {3},
  pages = {682},
  doi = {10.3390/math11030682}
}
Kostenlos starten →