Optimizing Apache Spark MLlib: Predictive Performance of Large-Scale Models for Big Data Analytics
Algorithms · 01.02.2025 · 18 (2) · 74
DOI: 10.3390/a18020074 ↗Kurz gefasst
Fachartikel in der Zeitschrift Algorithms, online erschienen am 01.02.2025. Verfasst von Leonidas Theodorakopoulos und 2 weiteren Autor:innen. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In this study, we analyze the performance of the machine learning operators in Apache Spark MLlib for K-Means, Random Forest Regression, and Word2Vec. We used a multi-node Spark cluster along with collected detailed execution metrics computed from the data of diverse datasets and parameter settings. The data were used to train predictive models that had up to 98% accuracy in forecasting performance. By building actionable predictive models, our research provides a unique treatment for key hyperparameter tuning, scalability, and real-time resource allocation challenges. Specifically, the practical value of traditional models in optimizing Apache Spark MLlib workflows was shown, achieving up to 30% resource savings and a 25% reduction in processing time. These models enable system optimization, reduce the amount of computational overheads, and boost the overall performance of big data applications. Ultimately, this work not only closes significant gaps in predictive performance modeling, but also paves the way for real-time analytics over a distributed environment.
Zitieren
APA
Theodorakopoulos, L., Karras, A., & Krimpas, G. A. (2025). Optimizing Apache Spark MLlib: Predictive Performance of Large-Scale Models for Big Data Analytics. Algorithms, 18(2), 74. https://doi.org/10.3390/a18020074
Vancouver
Theodorakopoulos L, Karras A, Krimpas GA. Optimizing Apache Spark MLlib: Predictive Performance of Large-Scale Models for Big Data Analytics. Algorithms. 2025;18(2):74. doi:10.3390/a18020074
BibTeX
@article{Theodorakopoulos2025,
author = {Theodorakopoulos, Leonidas and Karras, Aristeidis and Krimpas, George A.},
title = {Optimizing Apache Spark MLlib: Predictive Performance of Large-Scale Models for Big Data Analytics},
journal = {Algorithms},
year = {2025},
volume = {18},
number = {2},
pages = {74},
doi = {10.3390/a18020074}
}
EUVIMED