EUVIMEDEUVIMED entdecken
Crossref

Adaptive Biased Stochastic Optimization

Zhuang Yang

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-04 · 47 (4) · 3067-3078

DOI: 10.1109/tpami.2025.3528193 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-04. Verfasst von Zhuang Yang. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Jiangsu Province in China, Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This work develops and analyzes a class of adaptive biased stochastic optimization (ABSO) algorithms from the perspective of the GEneralized Adaptive gRadient (GEAR) method that contains Adam, AdaGrad, RMSProp, etc. Particularly, two preferred biased stochastic optimization (BSO) algorithms, the biased stochastic variance reduction gradient (BSVRG) algorithm and the stochastic recursive gradient algorithm (SARAH), equipped with GEAR, are first considered in this work, leading to two ABSO algorithms: BSVRG-GEAR and SARAH-GEAR. We present a uniform analysis of ABSO algorithms for minimizing strongly convex (SC) and Polyak-Łojasiewicz (PŁ) composite objective functions. Second, we also use our framework to develop another novel BSO algorithm, adaptive biased stochastic conjugate gradient (coined BSCG-GEAR), which achieves the well-known oracle complexity. Specifically, under mild conditions, we prove that the resulting ABSO algorithms attain a linear convergence rate on both PŁ and SC cases. Moreover, we show that the complexity of the resulting ABSO algorithms is comparable to that of advanced stochastic gradient-based algorithms. Finally, we demonstrate the empirical superiority and the numerical stability of the resulting ABSO algorithms by conducting numerical experiments on different applications of machine learning.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Jiangsu Province in China
  • Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions

Zitieren

APA

Yang, Z. (2025). Adaptive Biased Stochastic Optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4), 3067-3078. https://doi.org/10.1109/tpami.2025.3528193

Vancouver

Yang Z. Adaptive Biased Stochastic Optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(4):3067-3078. doi:10.1109/tpami.2025.3528193

BibTeX
@article{Yang2025,
  author = {Yang, Zhuang},
  title = {Adaptive Biased Stochastic Optimization},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {4},
  pages = {3067--3078},
  doi = {10.1109/tpami.2025.3528193}
}
Kostenlos starten →