Operationalizing MLOps with Databricks Pipelines: Scalable Machine Learning in Cloud Environments
International Journal of Scientific Research in Computer Science, Engineering and Information Technology · 25.12.2024 · 10 (6) · 2544-2552
DOI: 10.32628/cseit25113573 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Scientific Research in Computer Science, Engineering and Information Technology, online erschienen am 25.12.2024. Verfasst von Uttama Reddy Sanepalli. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 18 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The operationalization of machine learning models at scale remains a central challenge for data-driven enterprises due to complexities in deployment automation, governance, and post-deployment performance management. This article presents a structured MLOps framework leveraging Databricks pipelines to enable scalable, secure, and continuously monitored machine learning model deployment in cloud environments. The proposed approach integrates Delta Lake and Delta Live Tables for reliable data ingestion and transformation, MLflow for experiment tracking and model lifecycle management, and Databricks Model Serving for real-time and batch inference. The study demonstrates how automated pipelines support end-to-end orchestration of data preprocessing, distributed model training, hyperparameter optimization, deployment, and continuous performance monitoring. Built-in observability mechanisms and drift detection techniques are employed to ensure sustained model accuracy and reliability in production. By utilizing cloud-native infrastructure across AWS and Azure, the framework enhances scalability, fault tolerance, and operational efficiency while reducing manual intervention. The results highlight measurable improvements in deployment speed, model governance, and system reliability, underscoring the effectiveness of Databricks-based MLOps pipelines for enterprise-grade machine learning systems.
Zitieren
APA
Sanepalli, U. R. (2024). Operationalizing MLOps with Databricks Pipelines: Scalable Machine Learning in Cloud Environments. International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 10(6), 2544-2552. https://doi.org/10.32628/cseit25113573
Vancouver
Sanepalli UR. Operationalizing MLOps with Databricks Pipelines: Scalable Machine Learning in Cloud Environments. International Journal of Scientific Research in Computer Science, Engineering and Information Technology. 2024;10(6):2544-2552. doi:10.32628/cseit25113573
BibTeX
@article{Sanepalli2024,
author = {Sanepalli, Uttama Reddy},
title = {Operationalizing MLOps with Databricks Pipelines: Scalable Machine Learning in Cloud Environments},
journal = {International Journal of Scientific Research in Computer Science, Engineering and Information Technology},
year = {2024},
volume = {10},
number = {6},
pages = {2544--2552},
doi = {10.32628/cseit25113573}
}
EUVIMED