EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A MCDM-Based Framework for Quantifying Reproducibility Readiness in Machine Learning Research

Paulina Leščinskaitė, Remigijus Paulavičius, Ernestas Filatovas

Mathematics · 01.05.2026 · 14 (9) · 1536

DOI: 10.3390/math14091536 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematics, online erschienen am 01.05.2026. Verfasst von Paulina Leščinskaitė und 2 weiteren Autor:innen. Die Arbeit nennt 55 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Council of Lithuania.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Machine Learning (ML) is increasingly used across scientific domains, raising concerns about the reproducibility of published results. Reproducibility is a fundamental principle of scientific research, yet many ML research works remain difficult to reproduce due to missing artifacts and insufficient reporting. This study addresses the lack of practical quantitative methods for assessing reproducibility in ML research by proposing a paper-level evaluation framework based on Multi-Criteria Decision Making (MCDM). Through a synthesis of theoretical and data-driven analyses, we identified seven key reproducibility criteria: data and code availability, the inclusion of a README and trained models, hyperparameter and training descriptions, and paper readability. These criteria are then aggregated into a unified quantitative score for ‘R1’ level reproducibility readiness using the Weighted Sum Model (WSM) and the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). In the baseline evaluation, equal criterion weights were employed, while the Analytic Hierarchy Process (AHP) was demonstrated as a practical approach for deriving context-dependent weights tailored to diverse stakeholder priorities. The framework was applied to an annotated set of 139 randomly sampled ML research papers and benchmarked against an existing reproducibility label, achieving an accuracy of 0.64, a precision of 0.40, and a recall of 0.70. These results demonstrate that MCDM provides a feasible, interpretable, and flexible foundation for quantifying reproducibility readiness, allowing the assessment to be adapted to different decision-making contexts through customized criterion weighting.

Förderung

  • Research Council of Lithuania

Zitieren

APA

Leščinskaitė, P., Paulavičius, R., & Filatovas, E. (2026). A MCDM-Based Framework for Quantifying Reproducibility Readiness in Machine Learning Research. Mathematics, 14(9), 1536. https://doi.org/10.3390/math14091536

Vancouver

Leščinskaitė P, Paulavičius R, Filatovas E. A MCDM-Based Framework for Quantifying Reproducibility Readiness in Machine Learning Research. Mathematics. 2026;14(9):1536. doi:10.3390/math14091536

BibTeX
@article{Leinskait2026,
  author = {Leščinskaitė, Paulina and Paulavičius, Remigijus and Filatovas, Ernestas},
  title = {A MCDM-Based Framework for Quantifying Reproducibility Readiness in Machine Learning Research},
  journal = {Mathematics},
  year = {2026},
  volume = {14},
  number = {9},
  pages = {1536},
  doi = {10.3390/math14091536}
}
Kostenlos starten →