EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Modyn: Data-Centric Machine Learning Pipeline Orchestration

Maximilian Böther, Ties Robroek, Viktor Gsteiger, Robin Holzinger, Xianzhe Ma, Pınar Tözün, Ana Klimovic

Proceedings of the ACM on Management of Data · 10.02.2025 · 3 (1) · 1-30

DOI: 10.1145/3709705 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Management of Data, online erschienen am 11.02.2025. Verfasst von Maximilian Böther und 6 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 124 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In real-world machine learning (ML) pipelines, datasets are continuously growing. Models must incorporate this new training data to improve generalization and adapt to potential distribution shifts. The cost of model retraining is proportional to how frequently the model is retrained and how much data it is trained on, which makes the naive approach of retraining from scratch each time impractical. We present Modyn, a data-centric end-to-end machine learning platform. Modyn's ML pipeline abstraction enables users to declaratively describe policies for continuously training a model on a growing dataset. Modyn pipelines allow users to apply data selection policies (to reduce the number of data points) and triggering policies (to reduce the number of trainings). Modyn executes and orchestrates these continuous ML training pipelines. The system is open-source and comes with an ecosystem of benchmark datasets, models, and tooling. We formally discuss how to measure the performance of ML pipelines by introducing the concept of composite models, enabling fair comparison of pipelines with different data selection and triggering policies. We empirically analyze how various data selection and triggering policies impact model accuracy, and also show that Modyn enables high throughput training with sample-level data selection.

Förderung

  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung

Zitieren

APA

Böther, M., Robroek, T., Gsteiger, V., Holzinger, R., Ma, X., Tözün, P., & Klimovic, A. (2025). Modyn: Data-Centric Machine Learning Pipeline Orchestration. Proceedings of the ACM on Management of Data, 3(1), 1-30. https://doi.org/10.1145/3709705

Vancouver

Böther M, Robroek T, Gsteiger V, Holzinger R, Ma X, Tözün P, et al. Modyn: Data-Centric Machine Learning Pipeline Orchestration. Proceedings of the ACM on Management of Data. 2025;3(1):1-30. doi:10.1145/3709705

BibTeX
@article{Bther2025,
  author = {Böther, Maximilian and Robroek, Ties and Gsteiger, Viktor and Holzinger, Robin and Ma, Xianzhe and Tözün, Pınar and Klimovic, Ana},
  title = {Modyn: Data-Centric Machine Learning Pipeline Orchestration},
  journal = {Proceedings of the ACM on Management of Data},
  year = {2025},
  volume = {3},
  number = {1},
  pages = {1--30},
  doi = {10.1145/3709705}
}
Kostenlos starten →