Adaptive Node-Oriented Data Placement for Heterogeneous Hadoop Clusters
IEEE Access · 2025 · 13 · 79548-79559
DOI: 10.1109/access.2025.3565759 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2025. Verfasst von Vishnu Prasad Verma und 2 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Modern society is experiencing a data explosion thanks to rapid IT development and the increasing intelligence of devices. The vast and complex data can be utilized to extract actionable insight using a big data processing framework. Hadoop is a popular big data processing framework on heterogeneous commodity hardware. While Hadoop offers a robust framework for large-scale, data-intensive tasks via its MapReduce paradigm, hardware heterogeneity across nodes often leads to straggler effects that degrade Hadoop cluster performance. This paper introduces the Adaptive Node-Oriented Data placement for Efficient Hadoop Execution (ANODE) method, which leverages historical job execution data to dynamically assess each node’s processing capability. By employing an agent-based mechanism, ANODE optimizes block allocation within the data node, alleviating imbalances caused by Hadoop’s default uniform placement strategy. Experimental results on a heterogeneous eleven-node Hadoop cluster demonstrate that ANODE reduces job completion times by up to 25%, significantly enhancing data locality and resource utilization compared to the default approach.
Zitieren
APA
Verma, V. P., Kumar, S., & Naik, N. S. (2025). Adaptive Node-Oriented Data Placement for Heterogeneous Hadoop Clusters. IEEE Access, 13, 79548-79559. https://doi.org/10.1109/access.2025.3565759
Vancouver
Verma VP, Kumar S, Naik NS. Adaptive Node-Oriented Data Placement for Heterogeneous Hadoop Clusters. IEEE Access. 2025;13:79548-79559. doi:10.1109/access.2025.3565759
BibTeX
@article{Verma2025,
author = {Verma, Vishnu Prasad and Kumar, Santosh and Naik, Nenavath Srinivas},
title = {Adaptive Node-Oriented Data Placement for Heterogeneous Hadoop Clusters},
journal = {IEEE Access},
year = {2025},
volume = {13},
pages = {79548--79559},
doi = {10.1109/access.2025.3565759}
}
EUVIMED