EUVIMEDEUVIMED entdecken
Crossref

AI-Driven Self-Healing ETL Pipeline Using AWS Cloud Services for Automated Data Processing

Girish Dashmukhe, Yash Vinod Bhat, Manasi Badhe, Vaishnavi Khande, Vishakha Patel

INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · 25.03.2026 · 10 (03) · 1-9

DOI: 10.55041/ijsrem58245 ↗

Kurz gefasst

Fachartikel in der Zeitschrift INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT, online erschienen am 25.03.2026. Verfasst von Girish Dashmukhe und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Modern enterprises ingest vast amounts of data from diverse sources, requiring robust ETL (Extract-Transform-Load) pipelines. Traditional pipelines often break due to data inconsistencies or infrastructure faults. This paper proposes a self-healing ETL pipeline architecture built on AWS serverless services (S3, Glue, Lambda, Step Functions, etc.) augmented with AI/ML for automatic error detection and recovery. In our design, raw data lands in Amazon S3, triggers AWS Glue jobs for transformation, and any failures are detected via anomaly detection (e.g. using AWS Glue Data Quality or a SageMaker-trained model). A Step Functions workflow coordinates the ETL tasks and can retry or invoke corrective routines when errors occur. We include the architecture diagram, workflow, methodology, and cost analysis. Experimental results from related work show that self-healing pipelines significantly improve uptime and reduce manual intervention. KEYWORDS Cloud Computing, AWS, ETL Pipeline, Serverless Architecture, Artificial Intelligence, Machine Learning, Data Transformation, Data Quality, Automation, Data Pipeline.

Zitieren

APA

Dashmukhe, G., Bhat, Y. V., Badhe, M., Khande, V., & Patel, V. (2026). AI-Driven Self-Healing ETL Pipeline Using AWS Cloud Services for Automated Data Processing. INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT, 10(03), 1-9. https://doi.org/10.55041/ijsrem58245

Vancouver

Dashmukhe G, Bhat YV, Badhe M, Khande V, Patel V. AI-Driven Self-Healing ETL Pipeline Using AWS Cloud Services for Automated Data Processing. INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT. 2026;10(03):1-9. doi:10.55041/ijsrem58245

BibTeX
@article{Dashmukhe2026,
  author = {Dashmukhe, Girish and Bhat, Yash Vinod and Badhe, Manasi and Khande, Vaishnavi and Patel, Vishakha},
  title = {AI-Driven Self-Healing ETL Pipeline Using AWS Cloud Services for Automated Data Processing},
  journal = {INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT},
  year = {2026},
  volume = {10},
  number = {03},
  pages = {1--9},
  doi = {10.55041/ijsrem58245}
}
Kostenlos starten →