EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Adaptive Node-Oriented Data Placement for Heterogeneous Hadoop Clusters

Vishnu Prasad Verma, Santosh Kumar, Nenavath Srinivas Naik

IEEE Access · 2025 · 13 · 79548-79559

DOI: 10.1109/access.2025.3565759 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2025. Verfasst von Vishnu Prasad Verma und 2 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Modern society is experiencing a data explosion thanks to rapid IT development and the increasing intelligence of devices. The vast and complex data can be utilized to extract actionable insight using a big data processing framework. Hadoop is a popular big data processing framework on heterogeneous commodity hardware. While Hadoop offers a robust framework for large-scale, data-intensive tasks via its MapReduce paradigm, hardware heterogeneity across nodes often leads to straggler effects that degrade Hadoop cluster performance. This paper introduces the Adaptive Node-Oriented Data placement for Efficient Hadoop Execution (ANODE) method, which leverages historical job execution data to dynamically assess each node’s processing capability. By employing an agent-based mechanism, ANODE optimizes block allocation within the data node, alleviating imbalances caused by Hadoop’s default uniform placement strategy. Experimental results on a heterogeneous eleven-node Hadoop cluster demonstrate that ANODE reduces job completion times by up to 25%, significantly enhancing data locality and resource utilization compared to the default approach.

Zitieren

APA

Verma, V. P., Kumar, S., & Naik, N. S. (2025). Adaptive Node-Oriented Data Placement for Heterogeneous Hadoop Clusters. IEEE Access, 13, 79548-79559. https://doi.org/10.1109/access.2025.3565759

Vancouver

Verma VP, Kumar S, Naik NS. Adaptive Node-Oriented Data Placement for Heterogeneous Hadoop Clusters. IEEE Access. 2025;13:79548-79559. doi:10.1109/access.2025.3565759

BibTeX
@article{Verma2025,
  author = {Verma, Vishnu Prasad and Kumar, Santosh and Naik, Nenavath Srinivas},
  title = {Adaptive Node-Oriented Data Placement for Heterogeneous Hadoop Clusters},
  journal = {IEEE Access},
  year = {2025},
  volume = {13},
  pages = {79548--79559},
  doi = {10.1109/access.2025.3565759}
}
Kostenlos starten →