EUVIMEDEUVIMED entdecken
Crossref

Gradient Descent with Time-Decaying Regularization for Training Linear Neural Networks

Sergio Isai Palomino-Resendiz, César Ulises Solís-Cervantes, Luis Alberto Cantera-Cantera, Jorge de Jesús Morales-Mercado, Diego Alonso Flores-Hernández

Computer Modeling in Engineering & Sciences · 2026 · 147 (1) · 1-10

DOI: 10.32604/cmes.2026.077726 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computer Modeling in Engineering & Sciences, 2026. Verfasst von Sergio Isai Palomino-Resendiz und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

: Many linear-in-parameters models arising in identification and control can be expressed as single-layer artificial neural networks (ANNs) with linear activation, enabling online learning via first-order optimization. In practice, however, standard gradient descent often exhibits slow convergence, large intermediate weights, and stagnation when the regressor data are ill-conditioned or computations are performed under finite precision. This paper proposes Gradient Descent with Time-Decaying Regularization (GD-TDR), a training algorithm that augments the quadratic loss with a regularization term whose weight decays exponentially in time. The proposed schedule enforces uniform strong convexity during early iterations, effectively mitigating neural-paralysis-like behavior associated with flat directions, while asymptotically vanishing so that the unregularized least-squares solution is recovered. A convergence theorem for GD-TDR is established and a concise pseudocode implementation is provided. Numerical and embedded experiments on an online identification problem of a Chua-type chaotic oscillator demonstrate that GD-TDR converges faster and avoids stagnation compared to standard gradient descent, without introducing the steady-state bias characteristic of fixed quadratic regularization.

Zitieren

APA

Palomino-Resendiz, S. I., Solís-Cervantes, C. U., Cantera-Cantera, L. A., Morales-Mercado, J. D. J., & Flores-Hernández, D. A. (2026). Gradient Descent with Time-Decaying Regularization for Training Linear Neural Networks. Computer Modeling in Engineering & Sciences, 147(1), 1-10. https://doi.org/10.32604/cmes.2026.077726

Vancouver

Palomino-Resendiz SI, Solís-Cervantes CU, Cantera-Cantera LA, Morales-Mercado JDJ, Flores-Hernández DA. Gradient Descent with Time-Decaying Regularization for Training Linear Neural Networks. Computer Modeling in Engineering & Sciences. 2026;147(1):1-10. doi:10.32604/cmes.2026.077726

BibTeX
@article{PalominoResendiz2026,
  author = {Palomino-Resendiz, Sergio Isai and Solís-Cervantes, César Ulises and Cantera-Cantera, Luis Alberto and Morales-Mercado, Jorge de Jesús and Flores-Hernández, Diego Alonso},
  title = {Gradient Descent with Time-Decaying Regularization for Training Linear Neural Networks},
  journal = {Computer Modeling in Engineering & Sciences},
  year = {2026},
  volume = {147},
  number = {1},
  pages = {1--10},
  doi = {10.32604/cmes.2026.077726}
}
Kostenlos starten →