EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Optimizing Apache Spark MLlib: Predictive Performance of Large-Scale Models for Big Data Analytics

Leonidas Theodorakopoulos, Aristeidis Karras, George A. Krimpas

Algorithms · 01.02.2025 · 18 (2) · 74

DOI: 10.3390/a18020074 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Algorithms, online erschienen am 01.02.2025. Verfasst von Leonidas Theodorakopoulos und 2 weiteren Autor:innen. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this study, we analyze the performance of the machine learning operators in Apache Spark MLlib for K-Means, Random Forest Regression, and Word2Vec. We used a multi-node Spark cluster along with collected detailed execution metrics computed from the data of diverse datasets and parameter settings. The data were used to train predictive models that had up to 98% accuracy in forecasting performance. By building actionable predictive models, our research provides a unique treatment for key hyperparameter tuning, scalability, and real-time resource allocation challenges. Specifically, the practical value of traditional models in optimizing Apache Spark MLlib workflows was shown, achieving up to 30% resource savings and a 25% reduction in processing time. These models enable system optimization, reduce the amount of computational overheads, and boost the overall performance of big data applications. Ultimately, this work not only closes significant gaps in predictive performance modeling, but also paves the way for real-time analytics over a distributed environment.

Zitieren

APA

Theodorakopoulos, L., Karras, A., & Krimpas, G. A. (2025). Optimizing Apache Spark MLlib: Predictive Performance of Large-Scale Models for Big Data Analytics. Algorithms, 18(2), 74. https://doi.org/10.3390/a18020074

Vancouver

Theodorakopoulos L, Karras A, Krimpas GA. Optimizing Apache Spark MLlib: Predictive Performance of Large-Scale Models for Big Data Analytics. Algorithms. 2025;18(2):74. doi:10.3390/a18020074

BibTeX
@article{Theodorakopoulos2025,
  author = {Theodorakopoulos, Leonidas and Karras, Aristeidis and Krimpas, George A.},
  title = {Optimizing Apache Spark MLlib: Predictive Performance of Large-Scale Models for Big Data Analytics},
  journal = {Algorithms},
  year = {2025},
  volume = {18},
  number = {2},
  pages = {74},
  doi = {10.3390/a18020074}
}
Kostenlos starten →