EUVIMEDEUVIMED entdecken
Crossref

Two Steps at a Time---Taking GAN Training in Stride with Tseng's Method

Axel Böhm, Michael Sedlmayer, Ernö Robert Csetnek, Radu Ioan Boţ

SIAM Journal on Mathematics of Data Science · 2022-06 · 4 (2) · 750-771

DOI: 10.1137/21m1420939 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Mathematics of Data Science, online erschienen am 23.06.2022. Verfasst von Axel Böhm und 3 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 55 Literaturangaben. Gefördert von: Vienna Graduate School on Computational Optimization, Austrian Science Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivated by the training of generative adversarial networks (GANs), we study methods for solving minimax problems with additional nonsmooth regularizers. We do so by employing monotone operator theory, in particular the forward-backward-forward method, which avoids the known issue of limit cycling by correcting each update by a second gradient evaluation and does so requiring fewer projection steps compared to the extragradient method in the presence of constraints. Furthermore, we propose a seemingly new scheme which recycles old gradients to mitigate the additional computational cost. In doing so we rediscover a known method, related to optimistic gradient descent ascent. For both schemes we prove novel convergence rates for convex-concave minimax problems via a unifying approach. The derived error bounds are in terms of the gap function for the ergodic iterates. For the deterministic and the stochastic problem we show a convergence rate of $\mathcal{O}({1}/{k})$ and $\mathcal{O}({1}/{\sqrt{k}})$, respectively. We complement our theoretical results with empirical improvements in the training of Wasserstein GANs on the CIFAR10 dataset.

Förderung

  • Vienna Graduate School on Computational Optimization
  • Austrian Science Fund

Zitieren

APA

Böhm, A., Sedlmayer, M., Csetnek, E. R., & Boţ, R. I. (2022). Two Steps at a Time---Taking GAN Training in Stride with Tseng's Method. SIAM Journal on Mathematics of Data Science, 4(2), 750-771. https://doi.org/10.1137/21m1420939

Vancouver

Böhm A, Sedlmayer M, Csetnek ER, Boţ RI. Two Steps at a Time---Taking GAN Training in Stride with Tseng's Method. SIAM Journal on Mathematics of Data Science. 2022;4(2):750-771. doi:10.1137/21m1420939

BibTeX
@article{Bhm2022,
  author = {Böhm, Axel and Sedlmayer, Michael and Csetnek, Ernö Robert and Boţ, Radu Ioan},
  title = {Two Steps at a Time---Taking GAN Training in Stride with Tseng's Method},
  journal = {SIAM Journal on Mathematics of Data Science},
  year = {2022},
  volume = {4},
  number = {2},
  pages = {750--771},
  doi = {10.1137/21m1420939}
}
Kostenlos starten →