EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Benchmarking community drug response prediction models: datasets, models, tools, and metrics for cross-dataset generalization analysis

Alexander Partin iD, Priyanka Vasanthakumari iD, Oleksandr Narykov iD, Andreas Wilke iD, Natasha Koussa iD, Sara E Jones iD, Yitan Zhu iD, Jamie C Overbeek iD, Rajeev Jain, Gayara Demini Fernando iD, Cesar Sanchez-Villalobos, Cristina Garcia-Cardona iD und 8 weitere

Briefings in Bioinformatics · 2026

DOI: 10.1093/bib/bbaf667 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Briefings in Bioinformatics, 2026. Verfasst von Alexander Partin und 19 weiteren Autor:innen. Beteiligte Einrichtung: Computing, Environment and Life Sciences, Argonne National Laboratory, 9700 S Cass Ave, Lemont, 60439 IL, United States. Hauptthemen laut MEDLINE: Benchmarking, Deep Learning, Machine Learning. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NCI-DOE Collaboration Program, NCI NIH HHS, NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep learning and machine learning models have shown promise in drug response prediction (DRP), yet their ability to generalize across datasets remains an open question, raising concerns about their real-world applicability. Due to the lack of standardized benchmarking approaches, model evaluations and comparisons often rely on inconsistent datasets and evaluation criteria, making it difficult to assess true predictive capabilities. In this work, we introduce a benchmarking framework for evaluating cross-dataset prediction generalization in DRP models. Our framework incorporates five publicly available drug screening datasets, seven standardized DRP models, and a scalable workflow for systematic evaluation. To assess model generalization, we introduce a set of evaluation metrics that quantify both absolute performance (e.g. predictive accuracy across datasets) and relative performance (e.g. performance drop compared to within-dataset results), enabling a more comprehensive assessment of model transferability. Our results reveal substantial performance drops when models are tested on unseen datasets, underscoring the importance of rigorous generalization assessments. While several models demonstrate relatively strong cross-dataset generalization, no single model consistently outperforms across all datasets. Furthermore, we identify CTRPv2 as the most effective source dataset for training, yielding higher generalization scores across target datasets. By sharing this standardized evaluation framework with the community, our study aims to establish a rigorous foundation for model comparison, and accelerate the development of robust DRP models for real-world applications.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

BenchmarkingDeep LearningMachine LearningHumansPredictive Learning ModelsPrediction Algorithms

Stichwörter der Autor:innen

cross-dataset generalizationcross-study analysisdeep learningdrug response predictionmodel benchmarkingprecision oncology

Autor:innen und Institutionen

  • Computing, Environment and Life Sciences, Argonne National Laboratory, 9700 S Cass Ave, Lemont, 60439 IL, United States.
  • Cancer Data Science Initiatives, Cancer Research Technology Program, Frederick National Laboratory for Cancer Research, 8560 Progress Drive, Frederick, 21701 MD, United States.
  • Department of Statistics, University of Nebraska-Lincoln, 3310 Holdrege St, Lincoln, 68583 NE, United States.
  • Department of Electrical and Computer Engineering, Texas Tech University, 910 Boston Ave, Lubbock, 79409 TX, United States.
  • Division of Computer, Computational and Statistical Sciences, Los Alamos National Laboratory, Los Alamos, 87545 NM, United States.
  • Department of Computer Science, The University of Chicago, 5730 S Ellis Ave, Chicago, 60637 IL, United States.

ORCID-Profile: Alexander Partin, Priyanka Vasanthakumari, Oleksandr Narykov, Andreas Wilke, Natasha Koussa, Sara E Jones, Yitan Zhu, Jamie C Overbeek, Gayara Demini Fernando, Cristina Garcia-Cardona, Jamaludin Mohd-Yusof, Nicholas Chia, Justin M Wozniak, Ranadip Pal, Rick L Stevens

Förderung

  • NCI-DOE Collaboration Program
  • NCI NIH HHS
  • NIH HHS
  • Department of Energy
  • National Cancer Institute
  • National Institutes of Health

Zitieren

APA

Partin, A., Vasanthakumari, P., Narykov, O., Wilke, A., Koussa, N., Jones, S. E., Zhu, Y., Overbeek, J. C., Jain, R., Fernando, G. D., Sanchez-Villalobos, C., Garcia-Cardona, C., Mohd-Yusof, J., Chia, N., Wozniak, J. M., Ghosh, S., Pal, R., Brettin, T. S., Weil, M. R., & Stevens, R. L. (2026). Benchmarking community drug response prediction models: datasets, models, tools, and metrics for cross-dataset generalization analysis. Briefings in Bioinformatics. https://doi.org/10.1093/bib/bbaf667

Vancouver

Partin A, Vasanthakumari P, Narykov O, Wilke A, Koussa N, Jones SE, et al. Benchmarking community drug response prediction models: datasets, models, tools, and metrics for cross-dataset generalization analysis. Briefings in Bioinformatics. 2026. doi:10.1093/bib/bbaf667

BibTeX
@article{Partin2026,
  author = {Partin, Alexander and Vasanthakumari, Priyanka and Narykov, Oleksandr and Wilke, Andreas and Koussa, Natasha and Jones, Sara E and Zhu, Yitan and Overbeek, Jamie C and Jain, Rajeev and Fernando, Gayara Demini and Sanchez-Villalobos, Cesar and Garcia-Cardona, Cristina and Mohd-Yusof, Jamaludin and Chia, Nicholas and Wozniak, Justin M and Ghosh, Souparno and Pal, Ranadip and Brettin, Thomas S and Weil, M Ryan and Stevens, Rick L},
  title = {Benchmarking community drug response prediction models: datasets, models, tools, and metrics for cross-dataset generalization analysis},
  journal = {Briefings in Bioinformatics},
  year = {2026},
  doi = {10.1093/bib/bbaf667}
}
Kostenlos starten →