EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

ReLU Neural Networks and Their Training

Ge Luo, Xugang Wang, Weizun Zhao, Sichen Tao, Zheng Tang

Mathematics · 2025-12-22 · 14 (1) · 39

DOI: 10.3390/math14010039 ↗

At a glance

Journal article in Mathematics, published online on 2025-12-22. Written by Ge Luo and 4 co-authors. Cited 9 times so far (Crossref); 26 references. The full text is freely available (open access, CC BY 4.0). Funded by: Special Fund for Provincial Basic Research.

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Among various activation functions, the Rectified Linear Unit (ReLU) has become the most widely adopted due to its computational simplicity and effectiveness in mitigating the vanishing-gradient problem. In this work, we investigate the advantages of employing ReLU as the activation function and establish its theoretical significance. Our analysis demonstrates that ReLU-based neural networks possess the universal approximation property. In addition, we provide a theoretical explanation for the phenomenon of neuron death in ReLU-based neural networks. We further validate the effectiveness of this explanation through empirical experiments.

Funding

  • Special Fund for Provincial Basic Research

Cite

APA

Luo, G., Wang, X., Zhao, W., Tao, S., & Tang, Z. (2025). ReLU Neural Networks and Their Training. Mathematics, 14(1), 39. https://doi.org/10.3390/math14010039

Vancouver

Luo G, Wang X, Zhao W, Tao S, Tang Z. ReLU Neural Networks and Their Training. Mathematics. 2025;14(1):39. doi:10.3390/math14010039

BibTeX
@article{Luo2025,
  author = {Luo, Ge and Wang, Xugang and Zhao, Weizun and Tao, Sichen and Tang, Zheng},
  title = {ReLU Neural Networks and Their Training},
  journal = {Mathematics},
  year = {2025},
  volume = {14},
  number = {1},
  pages = {39},
  doi = {10.3390/math14010039}
}
Start for free →