An overview of the activation functions used in deep learning algorithms
Journal of New Results in Science · 31.12.2021 · 10 (3) · 75-88
DOI: 10.54187/jnrs.1011739 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of New Results in Science, online erschienen am 31.12.2021. Verfasst von Serhat Kılıçarslan und 2 weiteren Autor:innen. Die Arbeit wurde bisher 69-mal zitiert (Crossref) und nennt 45 Literaturangaben.
Abstract (Originalsprache)
In deep learning models, the inputs to the network are processed using activation functions to generate the output corresponding to these inputs. Deep learning models are of particular importance in analyzing big data with numerous parameters and forecasting and are useful for image processing, natural language processing, object recognition, and financial forecasting. Also, in deep learning algorithms, activation functions have been developed by taking into account features such as performing the learning process in a healthy way, preventing excessive learning, increasing the accuracy performance, and reducing the computational cost. In this study, we present an overview of common and current activation functions used in deep learning algorithms. In the study, fixed and trainable activation functions are introduced. As fixed activation functions, sigmoid, hyperbolic tangent, ReLU, softplus and swish, and as trainable activation functions, LReLU, ELU, SELU and RSigELU are introduced.
Zitieren
APA
Kılıçarslan, S., Adem, K., & Çelik, M. (2021). An overview of the activation functions used in deep learning algorithms. Journal of New Results in Science, 10(3), 75-88. https://doi.org/10.54187/jnrs.1011739
Vancouver
Kılıçarslan S, Adem K, Çelik M. An overview of the activation functions used in deep learning algorithms. Journal of New Results in Science. 2021;10(3):75-88. doi:10.54187/jnrs.1011739
BibTeX
@article{Klarslan2021,
author = {Kılıçarslan, Serhat and Adem, Kemal and Çelik, Mete},
title = {An overview of the activation functions used in deep learning algorithms},
journal = {Journal of New Results in Science},
year = {2021},
volume = {10},
number = {3},
pages = {75--88},
doi = {10.54187/jnrs.1011739}
}
EUVIMED