EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

An Improved BGE-Adam Optimization Algorithm Based on Entropy Weighting and Adaptive Gradient Strategy

Yichuan Shao, Jiantao Wang, Haijing Sun, Hao Yu, Lei Xing, Qian Zhao, Le Zhang

Symmetry · 17.05.2024 · 16 (5) · 623

DOI: 10.3390/sym16050623 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Symmetry, online erschienen am 17.05.2024. Verfasst von Yichuan Shao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Liaoning Provincial Department of Education’s Higher Education Foundation Research Project (General Project), Shenyang University of Technology, Liaoning Provincial Department of Education Science “14th Five-Year Plan”, Ministry of Education’s “Chunhui Plan”.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper introduces an enhanced variant of the Adam optimizer—the BGE-Adam optimization algorithm—that integrates three innovative technologies to augment the adaptability, convergence, and robustness of the original algorithm under various training conditions. Firstly, the BGE-Adam algorithm incorporates a dynamic β parameter adjustment mechanism that utilizes the rate of gradient variations to dynamically adjust the exponential decay rates of the first and second moment estimates (β1 and β2), the adjustment of β1 and β2 is symmetrical, which means that the rules that the algorithm considers when adjusting β1 and β2 are the same. This design helps to maintain the consistency and balance of the algorithm, allowing the optimization algorithm to adaptively capture the trending movements of gradients. Secondly, it estimates the direction of future gradients by a simple gradient prediction model, combining historic gradient information with the current gradient. Lastly, entropy weighting is integrated into the gradient update step. This strategy enhances the model’s exploratory nature by introducing a certain amount of noise, thereby improving its adaptability to complex loss surfaces. Experimental results on classical datasets, MNIST and CIFAR10, and gastrointestinal disease medical datasets demonstrate that the BGE-Adam algorithm has improved convergence and generalization capabilities. In particular, on the specific medical image gastrointestinal disease test dataset, the BGE-Adam optimization algorithm achieved an accuracy of 69.36%, a significant improvement over the 67.66% accuracy attained using the standard Adam algorithm; on the CIFAR10 test dataset, the accuracy of the BGE-Adam algorithm reached 71.4%, which is higher than the 70.65% accuracy of the Adam optimization algorithm; and on the MNIST dataset, the BGE-Adam algorithm’s accuracy was 99.34%, surpassing the Adam optimization algorithm’s accuracy of 99.23%. The BGE-Adam optimization algorithm exhibits better convergence and robustness. This research not only demonstrates the effectiveness of the combination of these three technologies but also provides new perspectives for the future development of deep learning optimization algorithms.

Förderung

  • Liaoning Provincial Department of Education’s Higher Education Foundation Research Project (General Project), Shenyang University of Technology
  • Liaoning Provincial Department of Education Science “14th Five-Year Plan”
  • Ministry of Education’s “Chunhui Plan”
  • Liaoning Provincial Department of Education’s Basic Research Project “Training and Application of Vertical Field Multi-Mode Deep Neural Network Model”
  • Shenyang Science and Technology Plan “Special Mission for Leech Breeding and Traditional Chinese Medicine Planting in Dengshibao Town, Faku County”

Zitieren

APA

Shao, Y., Wang, J., Sun, H., Yu, H., Xing, L., Zhao, Q., & Zhang, L. (2024). An Improved BGE-Adam Optimization Algorithm Based on Entropy Weighting and Adaptive Gradient Strategy. Symmetry, 16(5), 623. https://doi.org/10.3390/sym16050623

Vancouver

Shao Y, Wang J, Sun H, Yu H, Xing L, Zhao Q, et al. An Improved BGE-Adam Optimization Algorithm Based on Entropy Weighting and Adaptive Gradient Strategy. Symmetry. 2024;16(5):623. doi:10.3390/sym16050623

BibTeX
@article{Shao2024,
  author = {Shao, Yichuan and Wang, Jiantao and Sun, Haijing and Yu, Hao and Xing, Lei and Zhao, Qian and Zhang, Le},
  title = {An Improved BGE-Adam Optimization Algorithm Based on Entropy Weighting and Adaptive Gradient Strategy},
  journal = {Symmetry},
  year = {2024},
  volume = {16},
  number = {5},
  pages = {623},
  doi = {10.3390/sym16050623}
}
Kostenlos starten →