Accelerating Fixed-Point Algorithms in Statistics and Data Science: A State-of-Art Review
Journal of Data Science · 2023 · 1-26
DOI: 10.6339/22-jds1051 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Data Science, online erschienen am 19.07.2022. Verfasst von Bohao Tang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Fixed-point algorithms are popular in statistics and data science due to their simplicity, guaranteed convergence, and applicability to high-dimensional problems. Well-known examples include the expectation-maximization (EM) algorithm, majorization-minimization (MM), and gradient-based algorithms like gradient descent (GD) and proximal gradient descent. A characteristic weakness of these algorithms is their slow convergence. We discuss several state-of-art techniques for accelerating their convergence. We demonstrate and evaluate these techniques in terms of their efficiency and robustness in six distinct applications. Among the acceleration schemes, SQUAREM shows robust acceleration with a mean 18-fold speedup. DAAREM and restarted-Nesterov schemes also demonstrate consistently impressive accelerations. Thus, it is possible to accelerate the original fixed-point algorithm by using one of SQUAREM, DAAREM, or restarted-Nesterov acceleration schemes. We describe implementation details and software packages to facilitate the application of the acceleration schemes. We also discuss strategies for selecting a particular acceleration scheme for a given problem.
Zitieren
APA
Tang, B., Henderson, N. C., & Varadhan, R. (2023). Accelerating Fixed-Point Algorithms in Statistics and Data Science: A State-of-Art Review. Journal of Data Science, 1-26. https://doi.org/10.6339/22-jds1051
Vancouver
Tang B, Henderson NC, Varadhan R. Accelerating Fixed-Point Algorithms in Statistics and Data Science: A State-of-Art Review. Journal of Data Science. 2023:1-26. doi:10.6339/22-jds1051
BibTeX
@article{Tang2023,
author = {Tang, Bohao and Henderson, Nicholas C. and Varadhan, Ravi},
title = {Accelerating Fixed-Point Algorithms in Statistics and Data Science: A State-of-Art Review},
journal = {Journal of Data Science},
year = {2023},
pages = {1--26},
doi = {10.6339/22-jds1051}
}
EUVIMED