Minimax optimality of deep neural networks on dependent data via PAC-Bayes bounds
Electronic Journal of Statistics · 01.01.2025 · 19 (2) · 5895-5924
DOI: 10.1214/25-ejs2475 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronic Journal of Statistics, erschienen am 01.01.2025. Verfasst von Pierre Alquier und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 67 Literaturangaben.
Abstract (Originalsprache)
In a groundbreaking work, [58] proved the minimax optimality of deep neural networks with ReLU activation for least-squares regression estimation over a large class of functions defined by composition. In this paper, we extend these results in many directions. First, we remove the i.i.d. assumption on the observations, to allow some time dependence. The observations are assumed to be a Markov chain with a non-null pseudo-spectral gap. Then, we study a more general class of machine learning problems, which includes least-squares and logistic regression as special cases. Leveraging on PAC-Bayes oracle inequalities and a version of Bernstein inequality due to [53], we derive upper bounds on the estimation risk for a generalized Bayesian estimator. In the case of least-squares regression, this bound matches (up to a logarithmic factor) the lower bound in [58]. We establish a similar lower bound for classification with the logistic loss, and prove that the proposed DNN estimator is optimal in the minimax sense.
Zitieren
APA
Alquier, P., & Kengne, W. (2025). Minimax optimality of deep neural networks on dependent data via PAC-Bayes bounds. Electronic Journal of Statistics, 19(2), 5895-5924. https://doi.org/10.1214/25-ejs2475
Vancouver
Alquier P, Kengne W. Minimax optimality of deep neural networks on dependent data via PAC-Bayes bounds. Electronic Journal of Statistics. 2025;19(2):5895-5924. doi:10.1214/25-ejs2475
BibTeX
@article{Alquier2025,
author = {Alquier, Pierre and Kengne, William},
title = {Minimax optimality of deep neural networks on dependent data via PAC-Bayes bounds},
journal = {Electronic Journal of Statistics},
year = {2025},
volume = {19},
number = {2},
pages = {5895--5924},
doi = {10.1214/25-ejs2475}
}
EUVIMED