EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Incremental Online Learning of Randomized Neural Network With Forward Regularization

Junda Wang, Minghui Hu, Ning Li, Abdulaziz Al-Ali, Ponnuthurai Nagaratnam Suganthan

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-05 · 48 (5) · 5277-5293

DOI: 10.1109/tpami.2026.3652081 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-05. Verfasst von Junda Wang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 98 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, State Scholarship Fund of China Scholarship Council, Qatar National Library.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Online learning of deep neural networks faces challenges such as delayed non-incremental updating, increasing consumption, retrospective retraining, and catastrophic forgetting. To alleviate these drawbacks and achieve progressive immediate decision-making, we propose a novel Incremental Online Learning (IOL) framework of Randomized Neural Networks (Randomized NN), facilitating continuous improvements and analytics to Randomized NN performance in online scenarios. Within the framework, we further formulate IOL with ridge regularization (-R) and IOL with forward regularization (-F), both avoiding retrospective retraining and catastrophic forgetting. Moreover, the incremental algorithms for -R/-F on non-stationary batch stream are derived, featuring recursive weight updates and variable learning rates. Compared to -R, we recommend -F which improves learning performance using future unlabeled observations while further reducing online regrets to offline global experts. Additionally, we conduct a detailed analysis and theoretically derive relative cumulative regret bounds of the Randomized NN learners for -R/-F under adversarial assumptions via a novel methodology and present several corollaries, from which we observed the superiority in online learning acceleration and declined regret bounds of employing -F in IOL. Finally, our proposed methods were rigorously examined across diverse tasks, from simulation, regression, and classification tasks, to long-term time-series forecasting (LTSF) and continual learning (CL) fields, which distinctly validated the efficacy of the IOL frameworks and the advantages of forward regularization.

Förderung

  • National Natural Science Foundation of China
  • State Scholarship Fund of China Scholarship Council
  • Qatar National Library

Zitieren

APA

Wang, J., Hu, M., Li, N., Al-Ali, A., & Suganthan, P. N. (2026). Incremental Online Learning of Randomized Neural Network With Forward Regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(5), 5277-5293. https://doi.org/10.1109/tpami.2026.3652081

Vancouver

Wang J, Hu M, Li N, Al-Ali A, Suganthan PN. Incremental Online Learning of Randomized Neural Network With Forward Regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(5):5277-5293. doi:10.1109/tpami.2026.3652081

BibTeX
@article{Wang2026,
  author = {Wang, Junda and Hu, Minghui and Li, Ning and Al-Ali, Abdulaziz and Suganthan, Ponnuthurai Nagaratnam},
  title = {Incremental Online Learning of Randomized Neural Network With Forward Regularization},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {5},
  pages = {5277--5293},
  doi = {10.1109/tpami.2026.3652081}
}
Kostenlos starten →