Modified scaled exponential linear unit
Mathematics and Systems Science · 2024
DOI: 10.54517/mss.v2i2.2870 ↗Kurz gefasst
Fachartikel in der Zeitschrift Mathematics and Systems Science, 2024. Verfasst von Nimra Nimra und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Activation functions assume a crucial role in elucidating the intricacies of training dynamics and the overall performance of neural networks. Despite its simplicity and effectiveness, the ubiquitously embraced ReLU activation function harbors certain drawbacks, notably the predicament recognized as the “Dying ReLU” issue. To address such challenges, we propose the introduction of a pioneering activation function, the modified scaled exponential linear unit (M-SELU). Drawing from an array of experiments conducted across diverse computer vision tasks employing cutting-edge architectures, it becomes apparent that M-SELU exhibits superior performance compared to ReLU (used as the baseline) and various other activation functions. The simplicity of the proposed activation function (M-SELU) makes this solution particularly suitable for multi-layered deep neural architecture, including applications in CNN, CIFAR-10, and the broader field of deep learning.
Zitieren
APA
Nimra, N., Rahman, J. U., & Lu, D. (2024). Modified scaled exponential linear unit. Mathematics and Systems Science. https://doi.org/10.54517/mss.v2i2.2870
Vancouver
Nimra N, Rahman JU, Lu D. Modified scaled exponential linear unit. Mathematics and Systems Science. 2024. doi:10.54517/mss.v2i2.2870
BibTeX
@article{Nimra2024,
author = {Nimra, Nimra and Rahman, Jamshaid Ul and Lu, Dianchen},
title = {Modified scaled exponential linear unit},
journal = {Mathematics and Systems Science},
year = {2024},
doi = {10.54517/mss.v2i2.2870}
}
EUVIMED