EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A refined approach for evaluating small datasets via binary classification using machine learning

Steffen Steinert iD, Verena Ruf iD, David Dzsotjan, Nicolas Großmann, Albrecht Schmidt, Jochen Kuhn iD, Stefan Küchemann iD

PLOS ONE · 21.05.2024 · 19 (5) · e0301276

DOI: 10.1371/journal.pone.0301276 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 21.05.2024. Verfasst von Steffen Steinert und 6 weiteren Autor:innen. Beteiligte Einrichtung: Chair of Physics Education, Ludwig-Maximilians-Universität München (LMU Munich), Munich, Germany. Hauptthemen laut MEDLINE: Machine Learning. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Classical statistical analysis of data can be complemented or replaced with data analysis based on machine learning. However, in certain disciplines, such as education research, studies are frequently limited to small datasets, which raises several questions regarding biases and coincidentally positive results. In this study, we present a refined approach for evaluating the performance of a binary classification based on machine learning for small datasets. The approach includes a non-parametric permutation test as a method to quantify the probability of the results generalising to new data. Furthermore, we found that a repeated nested cross-validation is almost free of biases and yields reliable results that are only slightly dependent on chance. Considering the advantages of several evaluation metrics, we suggest a combination of more than one metric to train and evaluate machine learning classifiers. In the specific case that both classes are equally important, the Matthews correlation coefficient exhibits the lowest bias and chance for coincidentally good results. The results indicate that it is essential to avoid several biases when analysing small datasets using machine learning.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningHumansAlgorithmsDatasets as Topic

Autor:innen und Institutionen

  • Chair of Physics Education, Ludwig-Maximilians-Universität München (LMU Munich), Munich, Germany.
  • Department of Electrical and Computer Engineering, RPTU Kaiserslautern-Landau, Germany.
  • Smart Data & Knowledge Services, German Research Center for Artificial Intelligence, Kaiserslautern, Germany.
  • Human-Centered Ubiquitous Media, Ludwig-Maximilians-Universität München (LMU Munich), Munich, Germany.

ORCID-Profile: Steffen Steinert, Verena Ruf, Jochen Kuhn, Stefan Küchemann

Zitieren

APA

Steinert, S., Ruf, V., Dzsotjan, D., Großmann, N., Schmidt, A., Kuhn, J., & Küchemann, S. (2024). A refined approach for evaluating small datasets via binary classification using machine learning. PLOS ONE, 19(5), e0301276. https://doi.org/10.1371/journal.pone.0301276

Vancouver

Steinert S, Ruf V, Dzsotjan D, Großmann N, Schmidt A, Kuhn J, et al. A refined approach for evaluating small datasets via binary classification using machine learning. PLOS ONE. 2024;19(5):e0301276. doi:10.1371/journal.pone.0301276

BibTeX
@article{Steinert2024,
  author = {Steinert, Steffen and Ruf, Verena and Dzsotjan, David and Großmann, Nicolas and Schmidt, Albrecht and Kuhn, Jochen and Küchemann, Stefan},
  title = {A refined approach for evaluating small datasets via binary classification using machine learning},
  journal = {PLOS ONE},
  year = {2024},
  volume = {19},
  number = {5},
  pages = {e0301276},
  doi = {10.1371/journal.pone.0301276}
}
Kostenlos starten →