Zeroth-Order Gradient and Quasi-Newton Methods for Nonsmooth Nonconvex Stochastic Optimization
SIAM Journal on Optimization · 30.06.2026 · 36 (2) · 564-596
DOI: 10.1137/23m1626724 ↗Kurz gefasst
Fachartikel in der Zeitschrift SIAM Journal on Optimization, online erschienen am 02.04.2026. Verfasst von Luke Marrinan und 2 weiteren Autor:innen. Die Arbeit nennt 51 Literaturangaben. Gefördert von: Office of Naval Research, U.S. Department of Energy, AFOSR.
Abstract (Originalsprache)
We consider the minimization of a Lipschitz continuous and expectation-valued function, denoted by [Formula: see text] and defined as [Formula: see text], over a closed and convex set [Formula: see text]. Our focus lies on - both deriving asymptotics as well as obtaining rate and complexity guarantees for computing an approximate stationary point (in a Clarke sense) via zeroth-order schemes. We adopt a smoothing-based approach reliant on minimizing [Formula: see text], where [Formula: see text], [Formula: see text] is a random variable defined on a unit sphere, and [Formula: see text]. In fact, it has been observed that a stationary point of the [Formula: see text]-smoothed problem is an [Formula: see text]-stationary point for the original problem in the Clarke sense. In such a setting, we develop two sets of schemes with promising empirical behavior. (I) We develop a smoothing-enabled variance-reduced zeroth-order gradient framework for minimizing [Formula: see text] over [Formula: see text]. In this setting, we make two sets of contributions for the sequence generated by the proposed zeroth-order gradient scheme. (a) The residual function of the smoothed problem tends to zero almost surely along the generated sequence, allowing for making guarantees for [Formula: see text]-Clarke stationary solutions of the original problem. (b) To compute an [Formula: see text] that ensures that the expected norm of the residual of the [Formula: see text]-smoothed problem is within [Formula: see text] requires no greater than [Formula: see text] projection steps and [Formula: see text] function evaluations. (II) Our second scheme is a zeroth-order stochastic quasi-Newton scheme reliant on a combination of randomized and Moreau smoothing; the corresponding iteration and sample complexities for this scheme are [Formula: see text] and [Formula: see text], respectively. These statements appear to be novel in the case of both constrained problems as well as in stochastic quasi-Newton settings, as there appear to be few available results that can contend with general nonsmooth, nonconvex, and stochastic regimes via zeroth-order approaches.
Förderung
- Office of Naval Research
- U.S. Department of Energy
- AFOSR
Zitieren
APA
Marrinan, L., Shanbhag, U. V., & Yousefian, F. (2026). Zeroth-Order Gradient and Quasi-Newton Methods for Nonsmooth Nonconvex Stochastic Optimization. SIAM Journal on Optimization, 36(2), 564-596. https://doi.org/10.1137/23m1626724
Vancouver
Marrinan L, Shanbhag UV, Yousefian F. Zeroth-Order Gradient and Quasi-Newton Methods for Nonsmooth Nonconvex Stochastic Optimization. SIAM Journal on Optimization. 2026;36(2):564-596. doi:10.1137/23m1626724
BibTeX
@article{Marrinan2026,
author = {Marrinan, Luke and Shanbhag, Uday V. and Yousefian, Farzad},
title = {Zeroth-Order Gradient and Quasi-Newton Methods for Nonsmooth Nonconvex Stochastic Optimization},
journal = {SIAM Journal on Optimization},
year = {2026},
volume = {36},
number = {2},
pages = {564--596},
doi = {10.1137/23m1626724}
}
EUVIMED