Parallel PSO for Efficient Neural Network Training Using GPGPU and Apache Spark in Edge Computing Sets
Algorithms · 26.08.2024 · 17 (9) · 378
DOI: 10.3390/a17090378 ↗Kurz gefasst
Fachartikel in der Zeitschrift Algorithms, online erschienen am 26.08.2024. Verfasst von Manuel I. Capel und 2 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: panish Ministry of Science and Innovation (Ministerio de Ciencia e Innovación).
Abstract (Originalsprache)
The training phase of a deep learning neural network (DLNN) is a computationally demanding process, particularly for models comprising multiple layers of intermediate neurons.This paper presents a novel approach to accelerating DLNN training using the particle swarm optimisation (PSO) algorithm, which exploits the GPGPU architecture and the Apache Spark analytics engine for large-scale data processing tasks. PSO is a bio-inspired stochastic optimisation method whose objective is to iteratively enhance the solution to a (usually complex) problem by approximating a given objective. The expensive fitness evaluation and updating of particle positions can be supported more effectively by parallel processing. Nevertheless, the parallelisation of an efficient PSO is not a simple process due to the complexity of the computations performed on the swarm of particles and the iterative execution of the algorithm until a solution close to the objective with minimal error is achieved. In this study, two forms of parallelisation have been developed for the PSO algorithm, both of which are designed for execution in a distributed execution environment. The synchronous parallel PSO implementation guarantees consistency but may result in idle time due to global synchronisation. In contrast, the asynchronous parallel PSO approach reduces the necessity for global synchronization, thereby enhancing execution time and making it more appropriate for large datasets and distributed environments such as Apache Spark. The two variants of PSO have been implemented with the objective of distributing the computational load supported by the algorithm across the different executor nodes of the Spark cluster to effectively achieve coarse-grained parallelism. The result is a significant performance improvement over current sequential variants of PSO.
Förderung
- panish Ministry of Science and Innovation (Ministerio de Ciencia e Innovación)
Zitieren
APA
Capel, M. I., Salguero-Hidalgo, A., & Holgado-Terriza, J. A. (2024). Parallel PSO for Efficient Neural Network Training Using GPGPU and Apache Spark in Edge Computing Sets. Algorithms, 17(9), 378. https://doi.org/10.3390/a17090378
Vancouver
Capel MI, Salguero-Hidalgo A, Holgado-Terriza JA. Parallel PSO for Efficient Neural Network Training Using GPGPU and Apache Spark in Edge Computing Sets. Algorithms. 2024;17(9):378. doi:10.3390/a17090378
BibTeX
@article{Capel2024,
author = {Capel, Manuel I. and Salguero-Hidalgo, Alberto and Holgado-Terriza, Juan A.},
title = {Parallel PSO for Efficient Neural Network Training Using GPGPU and Apache Spark in Edge Computing Sets},
journal = {Algorithms},
year = {2024},
volume = {17},
number = {9},
pages = {378},
doi = {10.3390/a17090378}
}
EUVIMED