A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold
ACM Computing Surveys · 31.05.2025 · 57 (5) · 1-37
DOI: 10.1145/3708498 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 24.01.2025. Verfasst von Yanhong Fei und 5 weiteren Autor:innen. Die Arbeit wurde bisher 41-mal zitiert (Crossref) und nennt 203 Literaturangaben. Gefördert von: Natural Science Foundation of China, General Program of Shanghai Natural Science Foundation, Science and Technology Commission of Shanghai Municipality.
Abstract (Originalsprache)
Deep Learning (DL) has achieved remarkable success in tackling complex Artificial Intelligence tasks. The standard training of neural networks employs backpropagation to compute gradients and utilizes various optimization algorithms in the Euclidean space ℛ n . However, this optimization process faces challenges, such as the local optimal issues and the problem of gradient vanishing and exploding. To address these problems, Riemannian optimization offers a powerful extension to solve optimization problems in deep learning. By incorporating the prior constraint structure and the metric information of the underlying geometric information, Riemannian optimization-based DL offers a more stable and reliable optimization process, as well as enhanced adaptability to complex data structures. This article presents a comprehensive survey of applying geometric optimization in DL, including the basic procedure of geometric optimization, various geometric optimizers, and some concepts of the Riemannian manifold. In addition, it investigates various applications of geometric optimization in different DL networks for diverse tasks and discusses typical public toolboxes that implement optimization on the manifold. This article also includes a performance comparison among different deep geometric optimization methods in image recognition scenarios. Finally, this article elaborates on future opportunities and challenges in this field.
Förderung
- Natural Science Foundation of China
- General Program of Shanghai Natural Science Foundation
- Science and Technology Commission of Shanghai Municipality
- “Digital Silk Road” Shanghai International Joint Lab of Trustworthy Intelligent Software
- Shanghai Trusted Industry Internet Software Collaborative Innovation Center
Zitieren
APA
Fei, Y., Liu, Y., Jia, C., Li, Z., Wei, X., & Chen, M. (2025). A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold. ACM Computing Surveys, 57(5), 1-37. https://doi.org/10.1145/3708498
Vancouver
Fei Y, Liu Y, Jia C, Li Z, Wei X, Chen M. A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold. ACM Computing Surveys. 2025;57(5):1-37. doi:10.1145/3708498
BibTeX
@article{Fei2025,
author = {Fei, Yanhong and Liu, Yingjie and Jia, Chentao and Li, Zhengyu and Wei, Xian and Chen, Mingsong},
title = {A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold},
journal = {ACM Computing Surveys},
year = {2025},
volume = {57},
number = {5},
pages = {1--37},
doi = {10.1145/3708498}
}
EUVIMED