Container Profiler : Profiling resource utilization of containerized big data pipelines
GigaScience · 2022
DOI: 10.1093/gigascience/giad069 ↗Kurz gefasst
Fachartikel in der Zeitschrift GigaScience, 2022. Verfasst von Varik Hoang und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Engineering and Technology, University of Washington, Tacoma, WA 98402, USA. Hauptthemen laut MEDLINE: Big Data. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NIGMS NIH HHS, NIAID NIH HHS, NHGRI NIH HHS.
Abstract (Originalsprache)
Background
This article presents the Container Profiler, a software tool that measures and records the resource usage of any containerized task. Our tool profiles the CPU, memory, disk, and network utilization of containerized tasks collecting over 60 Linux operating system metrics at the virtual machine, container, and process levels. The Container Profiler supports performing time-series profiling at a configurable sampling interval to enable continuous monitoring of the resources consumed by containerized tasks and pipelines.
Results
To investigate the utility of the Container Profiler, we profile the resource utilization requirements of a multistage bioinformatics analytical pipeline (RNA sequencing using unique molecular identifiers). We examine profiling metrics to assess patterns of CPU, disk, and network resource utilization across the different stages of the pipeline. We also quantify the profiling overhead of our Container Profiler tool to assess the impact of profiling a running pipeline with different levels of profiling granularity, verifying that impacts are negligible.
Conclusions
The Container Profiler provides a useful tool that can be used to continuously monitor the resource consumption of long and complex containerized applications that run locally or on the cloud. This can help identify bottlenecks where more resources are needed to improve performance.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Engineering and Technology, University of Washington, Tacoma, WA 98402, USA.
ORCID-Profile: Varik Hoang, Ling-Hong Hung, Raymond Schooley, Ka Yee Yeung, Wes Lloyd
Förderung
- NIGMS NIH HHS
- NIAID NIH HHS
- NHGRI NIH HHS
- National Institutes of Health
Zitieren
APA
Hoang, V., Hung, L. H., Perez, D., Deng, H., Schooley, R., Arumilli, N., Yeung, K. Y., & Lloyd, W. (2022). Container Profiler : Profiling resource utilization of containerized big data pipelines. GigaScience. https://doi.org/10.1093/gigascience/giad069
Vancouver
Hoang V, Hung LH, Perez D, Deng H, Schooley R, Arumilli N, et al. Container Profiler : Profiling resource utilization of containerized big data pipelines. GigaScience. 2022. doi:10.1093/gigascience/giad069
BibTeX
@article{Hoang2022,
author = {Hoang, Varik and Hung, Ling-Hong and Perez, David and Deng, Huazeng and Schooley, Raymond and Arumilli, Niharika and Yeung, Ka Yee and Lloyd, Wes},
title = {Container Profiler : Profiling resource utilization of containerized big data pipelines},
journal = {GigaScience},
year = {2022},
doi = {10.1093/gigascience/giad069}
}
EUVIMED