EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

A comparative analysis of activation functions in neural networks: unveiling categories

Sara Bouraya, Abdessamad Belangour

Bulletin of Electrical Engineering and Informatics · 01.10.2024 · 13 (5) · 3301-3308

DOI: 10.11591/eei.v13i5.7274 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bulletin of Electrical Engineering and Informatics, online erschienen am 01.10.2024. Verfasst von Sara Bouraya und 1 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Activation functions (AFs) play a critical role in artificial neural networks, allowing for the modeling of complex, non-linear relationships in data. In this review paper, we provide an overview of the most commonly used AFs in deep learning. In this comparative study, we survey and compare the different AFs in deep learning and artificial neural networks. Our aim is to provide insights into the strengths and weaknesses of each AF and to provide guidance on the appropriate selection of AFs for different types of problems. We evaluate the most commonly used AFs, including sigmoid, tanh, rectified linear units (ReLUs) and its variants, exponential linear unit (ELU), and SoftMax. For each activation category, we discuss its properties, mathematical formulation (MF), and the benefits and drawbacks in terms of its ability to model complex, non-linear relationships in data. In conclusion, this comparative study provides a comprehensive overview of the properties and performance of different AFs, and serves as a valuable resource for researchers and practitioners in deep learning and artificial neural networks.

Zitieren

APA

Bouraya, S., & Belangour, A. (2024). A comparative analysis of activation functions in neural networks: unveiling categories. Bulletin of Electrical Engineering and Informatics, 13(5), 3301-3308. https://doi.org/10.11591/eei.v13i5.7274

Vancouver

Bouraya S, Belangour A. A comparative analysis of activation functions in neural networks: unveiling categories. Bulletin of Electrical Engineering and Informatics. 2024;13(5):3301-3308. doi:10.11591/eei.v13i5.7274

BibTeX
@article{Bouraya2024,
  author = {Bouraya, Sara and Belangour, Abdessamad},
  title = {A comparative analysis of activation functions in neural networks: unveiling categories},
  journal = {Bulletin of Electrical Engineering and Informatics},
  year = {2024},
  volume = {13},
  number = {5},
  pages = {3301--3308},
  doi = {10.11591/eei.v13i5.7274}
}
Kostenlos starten →