EUVIMEDEUVIMED entdecken
Crossref

Efficient Gradient-Based Methods for Bilevel Learning via Recycling Krylov Subspaces

M. J. Ehrhardt, Silvia Gazzola, Sebastian J. Scott

SIAM Journal on Scientific Computing · 30.06.2026 · 48 (3) · A1811-A1838

DOI: 10.1137/24m1716884 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Scientific Computing, online erschienen am 24.06.2026. Verfasst von M. J. Ehrhardt und 2 weiteren Autor:innen. Die Arbeit nennt 78 Literaturangaben. Gefördert von: Engineering and Physical Sciences Research Council Centre for Doctoral Training in Statistical Applied Mathematics at Bath, Bundesministerium für Bildung und Forschung, H2020 Marie Skłodowska-Curie Actions.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Many optimization problems require hyperparameters, i.e., parameters that must be prespecified in advance of their solution, such as regularization parameters and parametric regularizers in variational regularization methods for inverse problems and dictionaries in compressed sensing. A data-driven approach to determine appropriate hyperparameter values is via a nested optimization framework known as bilevel learning. Even when it is possible to employ a gradient-based solver to the bilevel optimization problem, construction of the gradients, known as hypergradients, is computationally challenging, each one requiring both a solution of a minimization problem and a linear system solve. These systems do not change much during the iterations, which motivates us to apply recycling Krylov subspace methods, wherein information from one linear system solve is reused to solve the next linear system. Existing recycling strategies often employ eigenvector approximations called Ritz vectors. In this work, we propose a novel recycling strategy based on a new concept, Ritz generalized singular vectors, which acknowledge the bilevel setting. Additionally, while existing iterative methods primarily terminate according to the residual norm, the newly introduced Ritz generalized singular vectors allow us to define a new stopping criterion that directly approximates the error of the associated hypergradient. The proposed approach is validated through extensive numerical testing in the context of inverse problems in imaging. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/s-j-scott/bilevel-recycling and in the supplementary materials ( M171688_Supplementary_Materials_1.pdf [177KB], bilevel-recycling-main.zip [80.4MB]). [Formula: see text]

Förderung

  • Engineering and Physical Sciences Research Council Centre for Doctoral Training in Statistical Applied Mathematics at Bath
  • Bundesministerium für Bildung und Forschung
  • H2020 Marie Skłodowska-Curie Actions
  • Engineering and Physical Sciences Research Council

Zitieren

APA

Ehrhardt, M. J., Gazzola, S., & Scott, S. J. (2026). Efficient Gradient-Based Methods for Bilevel Learning via Recycling Krylov Subspaces. SIAM Journal on Scientific Computing, 48(3), A1811-A1838. https://doi.org/10.1137/24m1716884

Vancouver

Ehrhardt MJ, Gazzola S, Scott SJ. Efficient Gradient-Based Methods for Bilevel Learning via Recycling Krylov Subspaces. SIAM Journal on Scientific Computing. 2026;48(3):A1811-A1838. doi:10.1137/24m1716884

BibTeX
@article{Ehrhardt2026,
  author = {Ehrhardt, M. J. and Gazzola, Silvia and Scott, Sebastian J.},
  title = {Efficient Gradient-Based Methods for Bilevel Learning via Recycling Krylov Subspaces},
  journal = {SIAM Journal on Scientific Computing},
  year = {2026},
  volume = {48},
  number = {3},
  pages = {A1811--A1838},
  doi = {10.1137/24m1716884}
}
Kostenlos starten →