Comparison of an effectiveness of artificial neural networks for various activation functions
Journal of Computer Sciences Institute · 30.03.2023 · 26 · 7-12
DOI: 10.35784/jcsi.3069 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Computer Sciences Institute, online erschienen am 30.03.2023. Verfasst von Daniel Florek und 1 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
Activation functions play an important role in artificial neural networks (ANNs) because they break the linearity in the data transformations that are performed by models. Thanks to the recent spike in interest around the topic of ANNs, new improvements to activation functions are emerging. The paper presents the results of research on the effectiveness of ANNs for ReLU, Leaky ReLU, ELU, and Swish activation functions. Four different data sets, and three different network architectures were used. Results show that Leaky ReLU, ELU and Swish functions work better in deep and more complex architectures which are to alleviate vanishing gradient and dead neurons problems but at the cost of prediction speed. Swish function seems to speed up training process considerably but neither of the three aforementioned functions comes ahead in accuracy in all used datasets.
Zitieren
APA
Florek, D., & Miłosz, M. (2023). Comparison of an effectiveness of artificial neural networks for various activation functions. Journal of Computer Sciences Institute, 26, 7-12. https://doi.org/10.35784/jcsi.3069
Vancouver
Florek D, Miłosz M. Comparison of an effectiveness of artificial neural networks for various activation functions. Journal of Computer Sciences Institute. 2023;26:7-12. doi:10.35784/jcsi.3069
BibTeX
@article{Florek2023,
author = {Florek, Daniel and Miłosz, Marek},
title = {Comparison of an effectiveness of artificial neural networks for various activation functions},
journal = {Journal of Computer Sciences Institute},
year = {2023},
volume = {26},
pages = {7--12},
doi = {10.35784/jcsi.3069}
}
EUVIMED