EUVIMEDEUVIMED entdecken
Crossref

Sharp Concentration Analysis of Online Stochastic Approximation Algorithms

Danna Zhang, Zhipeng Lou, Wei Biao Wu

IEEE Transactions on Information Theory · 2026-09 · 72 (9) · 6963-6975

DOI: 10.1109/tit.2026.3702077 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Information Theory, 2026-09. Verfasst von Danna Zhang und 2 weiteren Autor:innen. Die Arbeit nennt 74 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper introduces a comprehensive framework for analyzing online stochastic approximation algorithms from the perspective of nonlinear time series analysis. We introduce functional dependence measure to capture the underlying complex dependence structure of these algorithms, which can significantly facilitate the theoretical studies. In the more realistic setting where the gradient noise exhibits only a finite $q$ -th moment $(q \gt 1)$ , we develop a new Burkholder-type inequality as a powerful tool for analyzing general $q$ -th moments of online stochastic approximation algorithms, and systematically quantifying the proposed functional dependence measures. Equipped with these quantified dependence measures, we propose insightful and elegant approaches to derive sharp $q$ -th moment and tail probability inequalities for both the iterates of the stochastic gradient descent algorithm and the Polyak-Ruppert averaging estimators. In particular, to address the non-stationary nature of the outputs of these algorithms, we introduce a previously underexplored blocking technique with time-varying block sizes, which can be of independent interest when dealing with non-stationary processes.

Zitieren

APA

Zhang, D., Lou, Z., & Wu, W. B. (2026). Sharp Concentration Analysis of Online Stochastic Approximation Algorithms. IEEE Transactions on Information Theory, 72(9), 6963-6975. https://doi.org/10.1109/tit.2026.3702077

Vancouver

Zhang D, Lou Z, Wu WB. Sharp Concentration Analysis of Online Stochastic Approximation Algorithms. IEEE Transactions on Information Theory. 2026;72(9):6963-6975. doi:10.1109/tit.2026.3702077

BibTeX
@article{Zhang2026,
  author = {Zhang, Danna and Lou, Zhipeng and Wu, Wei Biao},
  title = {Sharp Concentration Analysis of Online Stochastic Approximation Algorithms},
  journal = {IEEE Transactions on Information Theory},
  year = {2026},
  volume = {72},
  number = {9},
  pages = {6963--6975},
  doi = {10.1109/tit.2026.3702077}
}
Kostenlos starten →