ReLU Neural Networks and Their Training
Mathematics · 22.12.2025 · 14 (1) · 39
DOI: 10.3390/math14010039 ↗Kurz gefasst
Fachartikel in der Zeitschrift Mathematics, online erschienen am 22.12.2025. Verfasst von Ge Luo und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Special Fund for Provincial Basic Research.
Abstract (Originalsprache)
Among various activation functions, the Rectified Linear Unit (ReLU) has become the most widely adopted due to its computational simplicity and effectiveness in mitigating the vanishing-gradient problem. In this work, we investigate the advantages of employing ReLU as the activation function and establish its theoretical significance. Our analysis demonstrates that ReLU-based neural networks possess the universal approximation property. In addition, we provide a theoretical explanation for the phenomenon of neuron death in ReLU-based neural networks. We further validate the effectiveness of this explanation through empirical experiments.
Förderung
- Special Fund for Provincial Basic Research
Zitieren
APA
Luo, G., Wang, X., Zhao, W., Tao, S., & Tang, Z. (2025). ReLU Neural Networks and Their Training. Mathematics, 14(1), 39. https://doi.org/10.3390/math14010039
Vancouver
Luo G, Wang X, Zhao W, Tao S, Tang Z. ReLU Neural Networks and Their Training. Mathematics. 2025;14(1):39. doi:10.3390/math14010039
BibTeX
@article{Luo2025,
author = {Luo, Ge and Wang, Xugang and Zhao, Weizun and Tao, Sichen and Tang, Zheng},
title = {ReLU Neural Networks and Their Training},
journal = {Mathematics},
year = {2025},
volume = {14},
number = {1},
pages = {39},
doi = {10.3390/math14010039}
}
EUVIMED