EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

AI Optimized Parallel Storage Orchestration for High Performance Computing Workloads in Hybrid Cloud Environments

Rahiman Shareef

Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023 · 22.01.2024 · 1 (1) · 303-309

DOI: 10.60087/jaigs.v1i1.463 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023, online erschienen am 22.01.2024. Verfasst von Rahiman Shareef. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

High Performance Computing workloads generate massive parallel input and output operations that place extreme demands on storage subsystems. Modern HPC environments increasingly integrate on-premises clusters with cloud infrastructure, creating hybrid architectures where data must move efficiently across heterogeneous storage tiers. Traditional parallel file systems rely on static striping, predefined caching policies, and manual performance tuning. These approaches are insufficient for dynamic workloads such as distributed AI training, simulation bursts, and checkpoint-intensive applications. This paper proposes an AI optimized parallel storage orchestration framework designed to enhance performance and resource utilization for HPC workloads in hybrid cloud environments. The framework monitors IO patterns, predicts workload behavior using lightweight machine learning models, and dynamically adjusts striping configurations, burst buffer usage, and cloud spillover policies. By introducing an intelligent orchestration layer above existing storage systems, the proposed approach improves adaptability without requiring modification of underlying file system architectures. A case study based evaluation demonstrates improved throughput stability during checkpoint storms, reduced IO contention during multi-tenant workloads, and better utilization of hybrid cloud storage resources compared to static configurations. The results indicate that AI driven orchestration can significantly enhance performance efficiency and scalability in next generation HPC storage systems.

Zitieren

APA

Shareef, R. (2024). AI Optimized Parallel Storage Orchestration for High Performance Computing Workloads in Hybrid Cloud Environments. Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023, 1(1), 303-309. https://doi.org/10.60087/jaigs.v1i1.463

Vancouver

Shareef R. AI Optimized Parallel Storage Orchestration for High Performance Computing Workloads in Hybrid Cloud Environments. Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023. 2024;1(1):303-309. doi:10.60087/jaigs.v1i1.463

BibTeX
@article{Shareef2024,
  author = {Shareef, Rahiman},
  title = {AI Optimized Parallel Storage Orchestration for High Performance Computing Workloads in Hybrid Cloud Environments},
  journal = {Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023},
  year = {2024},
  volume = {1},
  number = {1},
  pages = {303--309},
  doi = {10.60087/jaigs.v1i1.463}
}
Kostenlos starten →