Best Practices for Constructing, Preparing, and Evaluating Protein-Ligand Binding Affinity Benchmarks [Article v1.0]
Living Journal of Computational Molecular Science · 30.08.2022 · 4 (1) · 1497
DOI: 10.33011/livecoms.4.1.1497 ↗Kurz gefasst
Fachartikel in der Zeitschrift Living Journal of Computational Molecular Science, online erschienen am 30.08.2022. Verfasst von David F Hahn und 9 weiteren Autor:innen. Beteiligte Einrichtung: Computational Chemistry, Janssen Research & Development, Turnhoutseweg 30, Beerse B-2340, Belgium. Die Arbeit wurde bisher 78-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NCI NIH HHS, NIGMS NIH HHS.
Abstract (Originalsprache)
Free energy calculations are rapidly becoming indispensable in structure-enabled drug discovery programs. As new methods, force fields, and implementations are developed, assessing their accuracy on real-world systems (benchmarking) becomes critical to provide users with an assessment of the accuracy expected when these methods are applied within their domain of applicability, and developers with a way to assess the expected impact of new methodologies. These assessments require construction of a benchmark—a set of well-prepared, high quality systems with corresponding experimental measurements designed to ensure the resulting calculations provide a realistic assessment of expected performance. To date, the community has not yet adopted a common standardized benchmark, and existing benchmark reports suffer from a myriad of issues, including poor data quality, limited statistical power, and statistically deficient analyses, all of which can conspire to produce benchmarks that are poorly predictive of real-world performance. Here, we address these issues by presenting guidelines for (1) curating experimental data to develop meaningful benchmark sets, (2) preparing benchmark inputs according to best practices to facilitate widespread adoption, and (3) analysis of the resulting predictions to enable statistically meaningful comparisons among methods and force fields. We highlight challenges and open questions that remain to be solved in these areas, as well as recommendations for the collection of new datasets that might optimally serve to measure progress as methods become systematically more reliable. Finally, we provide a curated, versioned, open, standardized benchmark set adherent to these standards (protein-ligand-benchmark) and an open source toolkit for implementing standardized best practices assessments (arsenic) for the community to use as a standardized assessment tool. While our main focus is benchmarking free energy methods based on molecular simulations, these guidelines should prove useful for assessment of the rapidly growing field of machine learning methods for affinity prediction as well.
Autor:innen und Institutionen
- Computational Chemistry, Janssen Research & Development, Turnhoutseweg 30, Beerse B-2340, Belgium.
- OpenEye Scientific Software, 9 Bisbee Court, Suite D, Santa Fe, NM 87508 USA.
- Computational and Systems Biology Program, Sloan Kettering Institute, Memorial Sloan Kettering Cancer Center, New York, NY 10065 USA.
- MSD, The Francis Crick Institute, 1 Midland Road, London, NW1 1AT, United Kingdom.
- EaStCHEM School of Chemistry, David Brewster Road, Joseph Black Building, The King's Buildings, Edinburgh, EH9 3FJ, UK.
- Departments of Pharmaceutical Sciences and Chemistry, University of California, Irvine, CA USA.
ORCID-Profile: David F Hahn, Christopher I Bayly, Hannah E Bruce Macdonald, John D Chodera, Antonia S J S Mey, David L Mobley, Laura Perez Benito, Christina E M Schindler, Gary Tresadern, Gregory L Warren
Förderung
- NCI NIH HHS
- NIGMS NIH HHS
Zitieren
APA
Hahn, D. F., Bayly, C. I., Macdonald, H. E. B., Chodera, J. D., Mey, A. S. J. S., Mobley, D. L., Benito, L. P., Schindler, C. E. M., Tresadern, G., & Warren, G. L. (2022). Best Practices for Constructing, Preparing, and Evaluating Protein-Ligand Binding Affinity Benchmarks [Article v1.0]. Living Journal of Computational Molecular Science, 4(1), 1497. https://doi.org/10.33011/livecoms.4.1.1497
Vancouver
Hahn DF, Bayly CI, Macdonald HEB, Chodera JD, Mey ASJS, Mobley DL, et al. Best Practices for Constructing, Preparing, and Evaluating Protein-Ligand Binding Affinity Benchmarks [Article v1.0]. Living Journal of Computational Molecular Science. 2022;4(1):1497. doi:10.33011/livecoms.4.1.1497
BibTeX
@article{Hahn2022,
author = {Hahn, David F and Bayly, Christopher I and Macdonald, Hannah E Bruce and Chodera, John D and Mey, Antonia S J S and Mobley, David L and Benito, Laura Perez and Schindler, Christina E M and Tresadern, Gary and Warren, Gregory L},
title = {Best Practices for Constructing, Preparing, and Evaluating Protein-Ligand Binding Affinity Benchmarks [Article v1.0]},
journal = {Living Journal of Computational Molecular Science},
year = {2022},
volume = {4},
number = {1},
pages = {1497},
doi = {10.33011/livecoms.4.1.1497}
}
EUVIMED