Can Overfitted Deep Neural Networks in Adversarial Training Generalize? An Approximation Viewpoint
SIAM Journal on Mathematics of Data Science · 30.06.2026 · 8 (2) · 225-256
DOI: 10.1137/24m1634023 ↗Kurz gefasst
Fachartikel in der Zeitschrift SIAM Journal on Mathematics of Data Science, online erschienen am 22.04.2026. Verfasst von Zhongjie Shi und 3 weiteren Autor:innen. Die Arbeit nennt 54 Literaturangaben. Gefördert von: European Research Council.
Abstract (Originalsprache)
Adversarial training is a widely used method to improve the robustness of deep neural networks (DNNs) over adversarial perturbations. However, it is empirically observed that adversarial training on overparameterized networks often suffers from robust overfitting: it can achieve almost zero adversarial training error while the robust generalization performance is not promising. In this paper, we provide a theoretical understanding of the question of whether overfitted DNNs in adversarial training can generalize from an approximation viewpoint. Specifically, our main results are threefold: (i) For classification, we prove by construction the existence of infinitely many adversarial training classifiers on overparameterized DNNs that obtain arbitrarily small adversarial training error (overfitting) while achieving good robust generalization error under certain conditions concerning the data quality, data separation, and perturbation level. (ii) Linear overparameterization (meaning that the number of parameters is only slightly larger than the sample size) is enough to ensure such an existence if the target function is smooth enough. (iii) For regression, our results demonstrate that there also exist infinitely many overfitted DNNs with linear overparameterization in adversarial training that can achieve almost optimal rates of convergence for the standard generalization error. Overall, our analysis points out that robust overfitting can be avoided, but the required model capacity will depend on the smoothness of the target function, while a robust generalization gap is inevitable. We hope our analysis will give a better understanding of the mathematical foundations of robustness in DNNs from an approximation view.
Förderung
- European Research Council
Zitieren
APA
Shi, Z., Liu, F., Cao, Y., & Suykens, J. A. K. (2026). Can Overfitted Deep Neural Networks in Adversarial Training Generalize? An Approximation Viewpoint. SIAM Journal on Mathematics of Data Science, 8(2), 225-256. https://doi.org/10.1137/24m1634023
Vancouver
Shi Z, Liu F, Cao Y, Suykens JAK. Can Overfitted Deep Neural Networks in Adversarial Training Generalize? An Approximation Viewpoint. SIAM Journal on Mathematics of Data Science. 2026;8(2):225-256. doi:10.1137/24m1634023
BibTeX
@article{Shi2026,
author = {Shi, Zhongjie and Liu, Fanghui and Cao, Yuan and Suykens, Johan A. K.},
title = {Can Overfitted Deep Neural Networks in Adversarial Training Generalize? An Approximation Viewpoint},
journal = {SIAM Journal on Mathematics of Data Science},
year = {2026},
volume = {8},
number = {2},
pages = {225--256},
doi = {10.1137/24m1634023}
}
EUVIMED