EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Sublinearly structured deep neural networks achieve feature learning consistency for compositional functions

Sehwan Kim, Yan Sun, Faming Liang

Statistical Learning and Data Science · 2026

DOI: 10.1016/j.slads.2026.100007 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Statistical Learning and Data Science, 2026. Verfasst von Sehwan Kim und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Statistics, Ewha Womans University, Seoul 03760, Korea. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: NIGMS NIH HHS, National Institutes of Health, National Research Foundation of Korea.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models? While a full characterization is open, we provide positive results for a broad subclass. We establish feature-learning consistency guarantees for sublinearly structured DNNs-architectures whose input/output dimensions and number of hidden neurons grow sublinearly with the sample size-when learning hierarchically compositional target functions. Importantly, this consistency still holds even in the conventional "over-parameterized" regime where the total number of parameters exceeds the number of training samples. Empirically, sublinearly structured DNNs match or surpass wide DNNs in prediction. A structural audit further indicates that widely used convolutional neural networks, including AlexNet, VGGNet, ResNet, and GoogLeNet, are sublinearly structured on their image classification benchmarks. We further prove that the sublinearly structured DNNs achieve universal approximation for hierarchically compositional functions in the large-sample limit. Moreover, images exhibit an inherent hierarchical, compositional structure. Taken together, these results explain, through a statistical lens, why many large-scale deep learning models succeed after adequate training on massive image datasets.

Schlagwörter

Stichwörter der Autor:innen

62F1262M45Compositional functionEigen analysisFeature learning consistencyOver parameterizationStochastic neural network

Autor:innen und Institutionen

  • Department of Statistics, Ewha Womans University, Seoul 03760, Korea.
  • Department of Mathematical Science, New Jersey Institute of Technology, Newark, NJ 07102, USA.
  • Department of Statistics, Purdue University, West Lafayette, IN 47907, USA.

Förderung

  • NIGMS NIH HHS
  • National Institutes of Health
  • National Research Foundation of Korea
  • National Science Foundation

Zitieren

APA

Kim, S., Sun, Y., & Liang, F. (2026). Sublinearly structured deep neural networks achieve feature learning consistency for compositional functions. Statistical Learning and Data Science. https://doi.org/10.1016/j.slads.2026.100007

Vancouver

Kim S, Sun Y, Liang F. Sublinearly structured deep neural networks achieve feature learning consistency for compositional functions. Statistical Learning and Data Science. 2026. doi:10.1016/j.slads.2026.100007

BibTeX
@article{Kim2026,
  author = {Kim, Sehwan and Sun, Yan and Liang, Faming},
  title = {Sublinearly structured deep neural networks achieve feature learning consistency for compositional functions},
  journal = {Statistical Learning and Data Science},
  year = {2026},
  doi = {10.1016/j.slads.2026.100007}
}
Kostenlos starten →