Improved Convergence in Deep Neural Networks using a Modified Adaptive Moment Gradient Thresholding Algorithm
Faculty of Natural and Applied Sciences Journal of Computing and Applications · 2025
DOI: 10.63561/jca.v2i4.1069 ↗Kurz gefasst
Fachartikel in der Zeitschrift Faculty of Natural and Applied Sciences Journal of Computing and Applications, 2025. Verfasst von Mark Laisin und 3 weiteren Autor:innen.
Abstract (Originalsprache)
This study introduces the Adaptive Moment Gradient Thresholding (AMGT) algorithm, a modified version of the Adam optimizer, aimed at enhancing convergence stability in deep neural networks. By leveraging optimization theory and addressing the limitations of Adam, AMGT was designed to tackle non-convexity, constrained environments, and gradient-based learning instability. The algorithm incorporates a diminishing step size schedule and momentum thresholding to improve performance. Theoretical analysis demonstrated that AMGT achieved linear convergence under strong convexity with a rate of O(k^(-μ/2)), global convergence under bounded gradient approximation errors, and convergence to stationary points in non-convex scenarios. Numerical experiments on convex quadratic functions validated the theoretical predictions, highlighting the algorithm’s sensitivity to spectral properties and resilience to learning rate variations. The results indicate that AMGT surpasses standard Adam in convergence behaviour and provides theoretical guarantees often lacking in adaptive optimizers. AMGT is particularly effective in high-dimensional, noisy, or resource-constrained settings due to its support for quantized and sparsified updates. By combining theoretical rigour with empirical robustness, AMGT emerges as a dependable option for training deep learning models across diverse optimization landscapes.
Zitieren
APA
Laisin, M., Osu, B. O., Duruojinkeya, P. U., & Chibuisi, C. (2025). Improved Convergence in Deep Neural Networks using a Modified Adaptive Moment Gradient Thresholding Algorithm. Faculty of Natural and Applied Sciences Journal of Computing and Applications. https://doi.org/10.63561/jca.v2i4.1069
Vancouver
Laisin M, Osu BO, Duruojinkeya PU, Chibuisi C. Improved Convergence in Deep Neural Networks using a Modified Adaptive Moment Gradient Thresholding Algorithm. Faculty of Natural and Applied Sciences Journal of Computing and Applications. 2025. doi:10.63561/jca.v2i4.1069
BibTeX
@article{Laisin2025,
author = {Laisin, Mark and Osu, Bright Okore and Duruojinkeya, Prisca Udodiri and Chibuisi, Chigozie},
title = {Improved Convergence in Deep Neural Networks using a Modified Adaptive Moment Gradient Thresholding Algorithm},
journal = {Faculty of Natural and Applied Sciences Journal of Computing and Applications},
year = {2025},
doi = {10.63561/jca.v2i4.1069}
}
EUVIMED