EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Data management for distributed computational workflows: An iRODS-based setup and its performance

Mohamad Hayek iD, Martin Golasowski, Stephan Hachinger iD, Rubén J García-Hernández iD, Johannes Munke, Gabriel Lindner, Kateřina Slaninová iD, Philipp Tunka, Vít Vondrák, Dieter Kranzlmüller, Jan Martinovič iD

PLOS One · 2026

DOI: 10.1371/journal.pone.0340757 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2026. Verfasst von Mohamad Hayek und 10 weiteren Autor:innen. Beteiligte Einrichtung: Leibniz Supercomputing Centre (LRZ), Bavarian Academy of Sciences and Humanities, Garching near Munich, Germany. Hauptthemen laut MEDLINE: Workflow, Data Management. Die Arbeit nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: H2020 Industrial Leadership, HORIZON EUROPE Digital, Industry and Space, Bayerische Staatskanzlei.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Modern data-management frameworks promise a flexible and efficient management of data and metadata across storage backends. However, such claims need to be put to a meaningful test in daily practice. We conjecture that such frameworks should be fit to construct a data backend for workflows which use geographically distributed high-performance and cloud computing systems. Cross-site data transfers within such a backend should largely saturate network bandwidth, in particular when parameters such as buffer sizes are optimized. To explore this further, we evaluate the “integrated Rule-Oriented Data System” iRODS with EUDAT’s B2SAFE module as data backend for the “Distributed Data Infrastructure” within the LEXIS Platform for complex computing workflow orchestration and distributed data management. The focus of our study is on testing our conjectures—i.e., on construction and assessment of the data infrastructure and on measurements of data-transfer performance over the wide-area network between two selected supercomputing sites connected to LEXIS. We analyze limitations and identify optimization opportunities. Efficient utilization of the available network bandwidth is possible and depends on suitable client configuration and file size. Our work shows that systems such as iRODS nowadays fit the requirements for integration in federated computing infrastructures involving web-based authentication flows with OpenID Connect and rich on-line services. We are continuing to exploit these properties in the EXA4MIND project, where we aim at optimizing data-heavy workflows, integrating various systems for managing structured and unstructured data.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

WorkflowData ManagementSoftwareCloud Computing

Autor:innen und Institutionen

  • Leibniz Supercomputing Centre (LRZ), Bavarian Academy of Sciences and Humanities, Garching near Munich, Germany.
  • IT4Innovations National Supercomputing Center (IT4I), VŠB - Technical University of Ostrava, Ostrava, Czech Republic.
  • MNM-Team, Ludwig-Maximilians-Universität (LMU) München, Munich, Germany.

ORCID-Profile: Mohamad Hayek, Stephan Hachinger, Rubén J García-Hernández, Kateřina Slaninová, Jan Martinovič

Förderung

  • H2020 Industrial Leadership
  • HORIZON EUROPE Digital, Industry and Space
  • Bayerische Staatskanzlei
  • Ministerstvo Školství, Mládeže a Tělovýchovy

Zitieren

APA

Hayek, M., Golasowski, M., Hachinger, S., García-Hernández, R. J., Munke, J., Lindner, G., Slaninová, K., Tunka, P., Vondrák, V., Kranzlmüller, D., & Martinovič, J. (2026). Data management for distributed computational workflows: An iRODS-based setup and its performance. PLOS One. https://doi.org/10.1371/journal.pone.0340757

Vancouver

Hayek M, Golasowski M, Hachinger S, García-Hernández RJ, Munke J, Lindner G, et al. Data management for distributed computational workflows: An iRODS-based setup and its performance. PLOS One. 2026. doi:10.1371/journal.pone.0340757

BibTeX
@article{Hayek2026,
  author = {Hayek, Mohamad and Golasowski, Martin and Hachinger, Stephan and García-Hernández, Rubén J and Munke, Johannes and Lindner, Gabriel and Slaninová, Kateřina and Tunka, Philipp and Vondrák, Vít and Kranzlmüller, Dieter and Martinovič, Jan},
  title = {Data management for distributed computational workflows: An iRODS-based setup and its performance},
  journal = {PLOS One},
  year = {2026},
  doi = {10.1371/journal.pone.0340757}
}
Kostenlos starten →