Curse of dimensionality in neural network optimization
Information and Inference: A Journal of the IMA · 16.04.2026 · 15 (2)
DOI: 10.1093/imaiai/iaag009 ↗Kurz gefasst
Fachartikel in der Zeitschrift Information and Inference: A Journal of the IMA, online erschienen am 18.04.2026. Verfasst von Sanghoon Na und 1 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: U.S. National Science Foundation, Office of Naval Research Award, Department of Energy.
Abstract (Originalsprache)
This paper demonstrates that when a shallow neural network with a Lipschitz continuous activation function is trained using either empirical or population risk to approximate a target function i.e. $r$ times continuously differentiable on $[0,1]^{d}$, the population risk may not decay at a rate faster than $t^{-\frac{4r}{d-2r}}$, where $t$ denotes the time parameter of the gradient flow dynamics. This result highlights the presence of the curse of dimensionality in the optimization computation required to achieve a desired accuracy. Instead of analyzing parameter evolution directly, the training dynamics are examined through the evolution of the parameter distribution under the 2-Wasserstein gradient flow. Furthermore, it is established that the curse of dimensionality persists when a locally Lipschitz continuous activation function is employed, where the Lipschitz constant in $[-x,x]$ is bounded by $O(x^\delta )$ for any $x \in \mathbb{R}$. In this scenario, the population risk is shown to decay at a rate no faster than $t^{-\frac{(4+2\delta )r}{d-2r}}$. Understanding how function smoothness influences the curse of dimensionality in neural network optimization theory is an important and underexplored direction that this work aims to address.
Förderung
- U.S. National Science Foundation
- Office of Naval Research Award
- Department of Energy
- Defense Advanced Research Projects Agency
Zitieren
APA
Na, S., & Yang, H. (2026). Curse of dimensionality in neural network optimization. Information and Inference: A Journal of the IMA, 15(2). https://doi.org/10.1093/imaiai/iaag009
Vancouver
Na S, Yang H. Curse of dimensionality in neural network optimization. Information and Inference: A Journal of the IMA. 2026;15(2). doi:10.1093/imaiai/iaag009
BibTeX
@article{Na2026,
author = {Na, Sanghoon and Yang, Haizhao},
title = {Curse of dimensionality in neural network optimization},
journal = {Information and Inference: A Journal of the IMA},
year = {2026},
volume = {15},
number = {2},
doi = {10.1093/imaiai/iaag009}
}
EUVIMED