Training activation functions for layer insertion: a polynomial regression approach
Peer Review · 04.07.2026 · 8 (7) · 144-163
DOI: 10.53660/prw-2976-5309 ↗Kurz gefasst
Fachartikel in der Zeitschrift Peer Review, online erschienen am 04.07.2026. Verfasst von Carlos Eduardo Nascimento Cajado und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Artificial neural networks (ANNs) are widely applied to complex problems; however, the selection of an ideal activation function (AF) remains a challenge, particularly due to the lack of consistent criteria for relating data characteristics to the most suitable function. Recent advances highlight the potential of trainable AFs that dynamically adjust during training to enhance network performance. Additionally, that adjustment can be used for the insertion of a new layer. This work presents a novel approach that utilizes polynomial regression as the foundation for developing a layer insertion method based on trainable AFs. The proposal aims to go beyond conventional weight adjustment by allowing activation functions to evolve in an optimized manner across different contexts, thereby expanding the learning capabilities and performance of deep neural networks. The proposed approach presents a novel method for layer insertion, which can integrate a new layer in a collaborative way, which means that the new layer does not degrade the knowledge already acquired by the previous layers and besides can improve the effectiveness and the impacts of network adaptability and performance, when compared to fixed AFs as additional hidden layers are introduced.
Zitieren
APA
Cajado, C. E. N., Viana, F. D. S., Soares, C. M. M. D. O. P., & Neto, A. D. A. (2026). Training activation functions for layer insertion: a polynomial regression approach. Peer Review, 8(7), 144-163. https://doi.org/10.53660/prw-2976-5309
Vancouver
Cajado CEN, Viana FDS, Soares CMMDOP, Neto ADA. Training activation functions for layer insertion: a polynomial regression approach. Peer Review. 2026;8(7):144-163. doi:10.53660/prw-2976-5309
BibTeX
@article{Cajado2026,
author = {Cajado, Carlos Eduardo Nascimento and Viana, Francisco dos Santos and Soares, Carlos Manuel Milheiro de Oliveira Pinto and Neto, Areolino De Almeida},
title = {Training activation functions for layer insertion: a polynomial regression approach},
journal = {Peer Review},
year = {2026},
volume = {8},
number = {7},
pages = {144--163},
doi = {10.53660/prw-2976-5309}
}
EUVIMED