EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

PID controller‐based adaptive gradient optimizer for deep neural networks

Mingjun Dai, Zelong Zhang, Xiong Lai, Xiaohui Lin, Hui Wang

IET Control Theory & Applications · 2023-10 · 17 (15) · 2032-2037

DOI: 10.1049/cth2.12404 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IET Control Theory & Applications, online erschienen am 26.11.2022. Verfasst von Mingjun Dai und 4 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Due to improper selection of gradient update direction or learning rate, SGD optimization algorithms for deep learning suffer from oscillation and slow convergence. Although Adam algorithm can adaptively adjust the update direction and learning rate at the same time, it still has the overshoot phenomenon, and hence suffers from wasting computing resources and slow convergence. In this work, the PID controller from the feedback control area is borrowed to re‐express the adaptive optimization algorithm (the Adam optimization algorithm is derived into the integral I component form) of deep learning. In order to alleviate the overshoot phenomenon and hence speed up the convergence of Adam, a complete adaptive PID optimizer (adaptive‐PID) is proposed by incorporating the proportional P and derivative D component. Extensive experiments on standard data sets verify that the proposed adaptive‐PID algorithm significantly outperforms Adam algorithm in terms of convergence rate and accuracy.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Dai, M., Zhang, Z., Lai, X., Lin, X., & Wang, H. (2023). PID controller‐based adaptive gradient optimizer for deep neural networks. IET Control Theory & Applications, 17(15), 2032-2037. https://doi.org/10.1049/cth2.12404

Vancouver

Dai M, Zhang Z, Lai X, Lin X, Wang H. PID controller‐based adaptive gradient optimizer for deep neural networks. IET Control Theory & Applications. 2023;17(15):2032-2037. doi:10.1049/cth2.12404

BibTeX
@article{Dai2023,
  author = {Dai, Mingjun and Zhang, Zelong and Lai, Xiong and Lin, Xiaohui and Wang, Hui},
  title = {PID controller‐based adaptive gradient optimizer for deep neural networks},
  journal = {IET Control Theory & Applications},
  year = {2023},
  volume = {17},
  number = {15},
  pages = {2032--2037},
  doi = {10.1049/cth2.12404}
}
Kostenlos starten →