EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

On the role of benchmarking data sets and simulations in method comparison studies

Sarah Friedrich iD, Tim Friede iD

Biometrical Journal · 2024-01 · 66 (1)

DOI: 10.1002/bimj.202200212 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Biometrical Journal, online erschienen am 21.02.2023. Verfasst von Sarah Friedrich und 1 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Mathematics, University of Augsburg, Augsburg, Germany. Hauptthemen laut MEDLINE: Benchmarking. Die Arbeit wurde bisher 38-mal zitiert (Crossref) und nennt 100 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deutsche Forschungsgemeinschaft.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Method comparisons are essential to provide recommendations and guidance for applied researchers, who often have to choose from a plethora of available approaches. While many comparisons exist in the literature, these are often not neutral but favor a novel method. Apart from the choice of design and a proper reporting of the findings, there are different approaches concerning the underlying data for such method comparison studies. Most manuscripts on statistical methodology rely on simulation studies and provide a single real‐world data set as an example to motivate and illustrate the methodology investigated. In the context of supervised learning, in contrast, methods are often evaluated using so‐called benchmarking data sets, that is, real‐world data that serve as gold standard in the community. Simulation studies, on the other hand, are much less common in this context. The aim of this paper is to investigate differences and similarities between these approaches, to discuss their advantages and disadvantages, and ultimately to develop new approaches to the evaluation of methods picking the best of both worlds. To this aim, we borrow ideas from different contexts such as mixed methods research and Clinical Scenario Evaluation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

BenchmarkingComputer Simulation

Stichwörter der Autor:innen

benchmarkingmachine learningneutral comparison studiessimulation studies

Autor:innen und Institutionen

  • Institute of Mathematics, University of Augsburg, Augsburg, Germany.
  • Centre for Advanced Analytics and Predictive Sciences, University of Augsburg, Augsburg, Germany.
  • Department of Medical Statistics, University Medical Center Göttingen, Humboldtallee, Göttingen, Germany.
  • DZHK (German Centre for Cardiovascular Research), Partner Site Göttingen, Göttingen, Germany.

ORCID-Profile: Sarah Friedrich, Tim Friede

Förderung

  • Deutsche Forschungsgemeinschaft

Zitieren

APA

Friedrich, S., & Friede, T. (2024). On the role of benchmarking data sets and simulations in method comparison studies. Biometrical Journal, 66(1). https://doi.org/10.1002/bimj.202200212

Vancouver

Friedrich S, Friede T. On the role of benchmarking data sets and simulations in method comparison studies. Biometrical Journal. 2024;66(1). doi:10.1002/bimj.202200212

BibTeX
@article{Friedrich2024,
  author = {Friedrich, Sarah and Friede, Tim},
  title = {On the role of benchmarking data sets and simulations in method comparison studies},
  journal = {Biometrical Journal},
  year = {2024},
  volume = {66},
  number = {1},
  doi = {10.1002/bimj.202200212}
}
Kostenlos starten →