A comparative analysis of activation functions in neural networks: unveiling categories
Bulletin of Electrical Engineering and Informatics · 01.10.2024 · 13 (5) · 3301-3308
DOI: 10.11591/eei.v13i5.7274 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bulletin of Electrical Engineering and Informatics, online erschienen am 01.10.2024. Verfasst von Sara Bouraya und 1 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
Activation functions (AFs) play a critical role in artificial neural networks, allowing for the modeling of complex, non-linear relationships in data. In this review paper, we provide an overview of the most commonly used AFs in deep learning. In this comparative study, we survey and compare the different AFs in deep learning and artificial neural networks. Our aim is to provide insights into the strengths and weaknesses of each AF and to provide guidance on the appropriate selection of AFs for different types of problems. We evaluate the most commonly used AFs, including sigmoid, tanh, rectified linear units (ReLUs) and its variants, exponential linear unit (ELU), and SoftMax. For each activation category, we discuss its properties, mathematical formulation (MF), and the benefits and drawbacks in terms of its ability to model complex, non-linear relationships in data. In conclusion, this comparative study provides a comprehensive overview of the properties and performance of different AFs, and serves as a valuable resource for researchers and practitioners in deep learning and artificial neural networks.
Zitieren
APA
Bouraya, S., & Belangour, A. (2024). A comparative analysis of activation functions in neural networks: unveiling categories. Bulletin of Electrical Engineering and Informatics, 13(5), 3301-3308. https://doi.org/10.11591/eei.v13i5.7274
Vancouver
Bouraya S, Belangour A. A comparative analysis of activation functions in neural networks: unveiling categories. Bulletin of Electrical Engineering and Informatics. 2024;13(5):3301-3308. doi:10.11591/eei.v13i5.7274
BibTeX
@article{Bouraya2024,
author = {Bouraya, Sara and Belangour, Abdessamad},
title = {A comparative analysis of activation functions in neural networks: unveiling categories},
journal = {Bulletin of Electrical Engineering and Informatics},
year = {2024},
volume = {13},
number = {5},
pages = {3301--3308},
doi = {10.11591/eei.v13i5.7274}
}
EUVIMED