Prelimit Coupling and Steady-State Convergence of Constant-Step-Size Nonsmooth Contractive Stochastic Approximation
Operations Research · 2026-11 · 74 (6) · 3342-3358
DOI: 10.1287/opre.2024.1538 ↗Kurz gefasst
Fachartikel in der Zeitschrift Operations Research, 2026-11. Verfasst von Yixuan Zhang und 3 weiteren Autor:innen. Die Arbeit nennt 46 Literaturangaben.
Abstract (Originalsprache)
Nonsmooth Learning Algorithms Behave Differently from Smooth Ones Many learning algorithms, including Q-learning, update their estimates through noisy recursive rules. When these updates use a constant step size and involve nonsmooth operators, their long-run behavior can be difficult to characterize: the iterates do not converge to a single point but instead settle into a stationary distribution, and classical tools for smooth dynamics may fail. In “Prelimit Coupling and Steady-State Convergence of Constant-Step-Size Nonsmooth Contractive SA,” Yixuan Zhang, Dongyan (Lucy) Huo, Yudong Chen, and Qiaomin Xie develop a prelimit coupling framework for analyzing such nonsmooth stochastic approximation procedures. They establish Wasserstein convergence to a unique stationary distribution, characterize the limiting steady-state distribution as the step size vanishes, and show that nonsmoothness can create a bias of order of the square root of the step size, unlike the bias linear in the step size that is typical of smooth dynamics. The results also justify Richardson–Romberg extrapolation as a tool for bias reduction, with applications to Q-learning.
Zitieren
APA
Zhang, Y., Huo, D. L., Chen, Y., & Xie, Q. (2026). Prelimit Coupling and Steady-State Convergence of Constant-Step-Size Nonsmooth Contractive Stochastic Approximation. Operations Research, 74(6), 3342-3358. https://doi.org/10.1287/opre.2024.1538
Vancouver
Zhang Y, Huo DL, Chen Y, Xie Q. Prelimit Coupling and Steady-State Convergence of Constant-Step-Size Nonsmooth Contractive Stochastic Approximation. Operations Research. 2026;74(6):3342-3358. doi:10.1287/opre.2024.1538
BibTeX
@article{Zhang2026,
author = {Zhang, Yixuan and Huo, Dongyan (Lucy) and Chen, Yudong and Xie, Qiaomin},
title = {Prelimit Coupling and Steady-State Convergence of Constant-Step-Size Nonsmooth Contractive Stochastic Approximation},
journal = {Operations Research},
year = {2026},
volume = {74},
number = {6},
pages = {3342--3358},
doi = {10.1287/opre.2024.1538}
}
EUVIMED