EUVIMEDEUVIMED entdecken
Crossref

Self-Optimizing Pipelines ML Systems That Tune Themselves in Production

Sandeep Sarngadharan

International Journal of Computer Technology and Electronics Communication ( · 18.03.2025 · 08 (02)

DOI: 10.15680/ijctece.2025.0802015 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Technology and Electronics Communication (, online erschienen am 18.03.2025. Verfasst von Sandeep Sarngadharan.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Production machine learning pipelines are typically built, trained, and tested using historical data, then deployed with a fixed set of settings. However, real-world environments are constantly changing. Customer behavior evolves, data patterns shift, and system resources fluctuate. When performance drops, data scientists must manually investigate the problem, retune the model, and redeploy — a process that is slow, expensive, and prone to human error. This article introduces Self-Optimizing Pipelines (SOPs) — machine learning systems that automatically adjust their own configuration while running in production, without any human intervention. An SOP continuously monitors its own performance, detects when the data distribution has changed, searches for better settings (such as hyperparameters, feature selections, and ensemble weights), and safely deploys those improvements. We present a complete system architecture with five interconnected modules. Using a real-world credit card fraud detection dataset with simulated concept drift, we show that an SOP achieves an average F1-score of 0.93 compared to 0.71 for a static pipeline — a 31% improvement. The SOP reduces the need for manual data scientist intervention by 94% and adds only a small increase in processing time (from 12 milliseconds to 16 milliseconds per prediction), which is acceptable for most real-time applications. Our results demonstrate that self-tuning production systems are not only feasible but also stable, maintainable, and ready for deployment with existing open-source tools.

Zitieren

APA

Sarngadharan, S. (2025). Self-Optimizing Pipelines ML Systems That Tune Themselves in Production. International Journal of Computer Technology and Electronics Communication (, 08(02). https://doi.org/10.15680/ijctece.2025.0802015

Vancouver

Sarngadharan S. Self-Optimizing Pipelines ML Systems That Tune Themselves in Production. International Journal of Computer Technology and Electronics Communication (. 2025;08(02). doi:10.15680/ijctece.2025.0802015

BibTeX
@article{Sarngadharan2025,
  author = {Sarngadharan, Sandeep},
  title = {Self-Optimizing Pipelines ML Systems That Tune Themselves in Production},
  journal = {International Journal of Computer Technology and Electronics Communication (},
  year = {2025},
  volume = {08},
  number = {02},
  doi = {10.15680/ijctece.2025.0802015}
}
Kostenlos starten →