Adaptive-Tanh ReLU (AHReLU): A Novel Activation Function for Enhanced Convolutional Neural Network Performance
Punjab University Journal of Mathematics · 13.11.2025 · 57 (5) · 591-611
DOI: 10.52280/gh9se897 ↗Kurz gefasst
Fachartikel in der Zeitschrift Punjab University Journal of Mathematics, online erschienen am 13.11.2025. Verfasst von Abaid Ullah und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In this work, we propose a novel activation function, AHReLU, which demonstrates superior performance compared to traditional activa tion functions such as ReLU, GELU, and Mish. Replacing ReLU with AHReLU in the VGG-16 model improved Top-1 classification accuracy by 0.79%,reaching97.57%. In the CIFAR-100dataset, AHReLUoutper formed ReLU by 0.32%, achieving a Top-1 accuracy of 59.82%. For theSVHN dataset, AHReLU achieved a mean accuracy of 95.38%, slightly surpassing ReLU’s performance of 95.36%. In machine translation tasks, specifically on the WMT 2014 dataset, AHReLU achieved a BLEU score of 27.5, which is 0.2 points higher than ReLU and 1.2 points higher than GELU. These results highlight that AHReLU, with its learnable parame ters, outperforms traditional activation functions, leading to better model performance across various datasets and tasks. The introduction of learn able parameters into the activation function is key to the observed im provements, making AHReLU a promising candidate to replace widely used activation functions such as ReLU, GELU, and Mish in deep learning models.
Zitieren
APA
Ullah, A., Imran, M., & Akram, S. (2025). Adaptive-Tanh ReLU (AHReLU): A Novel Activation Function for Enhanced Convolutional Neural Network Performance. Punjab University Journal of Mathematics, 57(5), 591-611. https://doi.org/10.52280/gh9se897
Vancouver
Ullah A, Imran M, Akram S. Adaptive-Tanh ReLU (AHReLU): A Novel Activation Function for Enhanced Convolutional Neural Network Performance. Punjab University Journal of Mathematics. 2025;57(5):591-611. doi:10.52280/gh9se897
BibTeX
@article{Ullah2025,
author = {Ullah, Abaid and Imran, M. and Akram, Saima},
title = {Adaptive-Tanh ReLU (AHReLU): A Novel Activation Function for Enhanced Convolutional Neural Network Performance},
journal = {Punjab University Journal of Mathematics},
year = {2025},
volume = {57},
number = {5},
pages = {591--611},
doi = {10.52280/gh9se897}
}
EUVIMED