Benchmarking community drug response prediction models: datasets, models, tools, and metrics for cross-dataset generalization analysis
Briefings in Bioinformatics · 2026
DOI: 10.1093/bib/bbaf667 ↗Kurz gefasst
Fachartikel in der Zeitschrift Briefings in Bioinformatics, 2026. Verfasst von Alexander Partin und 19 weiteren Autor:innen. Beteiligte Einrichtung: Computing, Environment and Life Sciences, Argonne National Laboratory, 9700 S Cass Ave, Lemont, 60439 IL, United States. Hauptthemen laut MEDLINE: Benchmarking, Deep Learning, Machine Learning. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NCI-DOE Collaboration Program, NCI NIH HHS, NIH HHS.
Abstract (Originalsprache)
Deep learning and machine learning models have shown promise in drug response prediction (DRP), yet their ability to generalize across datasets remains an open question, raising concerns about their real-world applicability. Due to the lack of standardized benchmarking approaches, model evaluations and comparisons often rely on inconsistent datasets and evaluation criteria, making it difficult to assess true predictive capabilities. In this work, we introduce a benchmarking framework for evaluating cross-dataset prediction generalization in DRP models. Our framework incorporates five publicly available drug screening datasets, seven standardized DRP models, and a scalable workflow for systematic evaluation. To assess model generalization, we introduce a set of evaluation metrics that quantify both absolute performance (e.g. predictive accuracy across datasets) and relative performance (e.g. performance drop compared to within-dataset results), enabling a more comprehensive assessment of model transferability. Our results reveal substantial performance drops when models are tested on unseen datasets, underscoring the importance of rigorous generalization assessments. While several models demonstrate relatively strong cross-dataset generalization, no single model consistently outperforms across all datasets. Furthermore, we identify CTRPv2 as the most effective source dataset for training, yielding higher generalization scores across target datasets. By sharing this standardized evaluation framework with the community, our study aims to establish a rigorous foundation for model comparison, and accelerate the development of robust DRP models for real-world applications.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Computing, Environment and Life Sciences, Argonne National Laboratory, 9700 S Cass Ave, Lemont, 60439 IL, United States.
- Cancer Data Science Initiatives, Cancer Research Technology Program, Frederick National Laboratory for Cancer Research, 8560 Progress Drive, Frederick, 21701 MD, United States.
- Department of Statistics, University of Nebraska-Lincoln, 3310 Holdrege St, Lincoln, 68583 NE, United States.
- Department of Electrical and Computer Engineering, Texas Tech University, 910 Boston Ave, Lubbock, 79409 TX, United States.
- Division of Computer, Computational and Statistical Sciences, Los Alamos National Laboratory, Los Alamos, 87545 NM, United States.
- Department of Computer Science, The University of Chicago, 5730 S Ellis Ave, Chicago, 60637 IL, United States.
ORCID-Profile: Alexander Partin, Priyanka Vasanthakumari, Oleksandr Narykov, Andreas Wilke, Natasha Koussa, Sara E Jones, Yitan Zhu, Jamie C Overbeek, Gayara Demini Fernando, Cristina Garcia-Cardona, Jamaludin Mohd-Yusof, Nicholas Chia, Justin M Wozniak, Ranadip Pal, Rick L Stevens
Förderung
- NCI-DOE Collaboration Program
- NCI NIH HHS
- NIH HHS
- Department of Energy
- National Cancer Institute
- National Institutes of Health
Zitieren
APA
Partin, A., Vasanthakumari, P., Narykov, O., Wilke, A., Koussa, N., Jones, S. E., Zhu, Y., Overbeek, J. C., Jain, R., Fernando, G. D., Sanchez-Villalobos, C., Garcia-Cardona, C., Mohd-Yusof, J., Chia, N., Wozniak, J. M., Ghosh, S., Pal, R., Brettin, T. S., Weil, M. R., & Stevens, R. L. (2026). Benchmarking community drug response prediction models: datasets, models, tools, and metrics for cross-dataset generalization analysis. Briefings in Bioinformatics. https://doi.org/10.1093/bib/bbaf667
Vancouver
Partin A, Vasanthakumari P, Narykov O, Wilke A, Koussa N, Jones SE, et al. Benchmarking community drug response prediction models: datasets, models, tools, and metrics for cross-dataset generalization analysis. Briefings in Bioinformatics. 2026. doi:10.1093/bib/bbaf667
BibTeX
@article{Partin2026,
author = {Partin, Alexander and Vasanthakumari, Priyanka and Narykov, Oleksandr and Wilke, Andreas and Koussa, Natasha and Jones, Sara E and Zhu, Yitan and Overbeek, Jamie C and Jain, Rajeev and Fernando, Gayara Demini and Sanchez-Villalobos, Cesar and Garcia-Cardona, Cristina and Mohd-Yusof, Jamaludin and Chia, Nicholas and Wozniak, Justin M and Ghosh, Souparno and Pal, Ranadip and Brettin, Thomas S and Weil, M Ryan and Stevens, Rick L},
title = {Benchmarking community drug response prediction models: datasets, models, tools, and metrics for cross-dataset generalization analysis},
journal = {Briefings in Bioinformatics},
year = {2026},
doi = {10.1093/bib/bbaf667}
}
EUVIMED