ReLU Neural Networks and Their Training
Mathematics · 2025-12-22 · 14 (1) · 39
DOI: 10.3390/math14010039 ↗At a glance
Journal article in Mathematics, published online on 2025-12-22. Written by Ge Luo and 4 co-authors. Cited 9 times so far (Crossref); 26 references. The full text is freely available (open access, CC BY 4.0). Funded by: Special Fund for Provincial Basic Research.
Abstract (original language)
Among various activation functions, the Rectified Linear Unit (ReLU) has become the most widely adopted due to its computational simplicity and effectiveness in mitigating the vanishing-gradient problem. In this work, we investigate the advantages of employing ReLU as the activation function and establish its theoretical significance. Our analysis demonstrates that ReLU-based neural networks possess the universal approximation property. In addition, we provide a theoretical explanation for the phenomenon of neuron death in ReLU-based neural networks. We further validate the effectiveness of this explanation through empirical experiments.
Funding
- Special Fund for Provincial Basic Research
Cite
APA
Luo, G., Wang, X., Zhao, W., Tao, S., & Tang, Z. (2025). ReLU Neural Networks and Their Training. Mathematics, 14(1), 39. https://doi.org/10.3390/math14010039
Vancouver
Luo G, Wang X, Zhao W, Tao S, Tang Z. ReLU Neural Networks and Their Training. Mathematics. 2025;14(1):39. doi:10.3390/math14010039
BibTeX
@article{Luo2025,
author = {Luo, Ge and Wang, Xugang and Zhao, Weizun and Tao, Sichen and Tang, Zheng},
title = {ReLU Neural Networks and Their Training},
journal = {Mathematics},
year = {2025},
volume = {14},
number = {1},
pages = {39},
doi = {10.3390/math14010039}
}
EUVIMED