UNIQUE: A Framework for Uncertainty Quantification Benchmarking
Journal of Chemical Information and Modeling · 25.11.2024 · 64 (22) · 8379-8386
DOI: 10.1021/acs.jcim.4c01578 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Chemical Information and Modeling, online erschienen am 14.11.2024. Verfasst von Jessica Lanini und 4 weiteren Autor:innen. Beteiligte Einrichtung: Novartis Biomedical Research, Novartis Campus, 4002 Basel, Switzerland. Hauptthemen laut MEDLINE: Machine Learning. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Novartis.
Abstract (Originalsprache)
Machine learning (ML) models have become key in decision-making for many disciplines, including drug discovery and medicinal chemistry. ML models are generally evaluated prior to their usage in high-stakes decisions, such as compound synthesis or experimental testing. However, no ML model is robust or predictive in all real-world scenarios. Therefore, uncertainty quantification (UQ) in ML predictions has gained importance in recent years. Many investigations have focused on developing methodologies that provide accurate uncertainty estimates for ML-based predictions. Unfortunately, there is no UQ strategy that consistently provides robust estimates about model's applicability on new samples. Depending on the dataset, prediction task, and algorithm, accurate uncertainty estimations might be unfeasible to obtain. Moreover, the optimum UQ metric also varies across applications, and previous investigations have shown a lack of consistency across benchmarks. Herein, the UNIQUE (UNcertaInty QUantification bEnchmarking) framework is introduced to facilitate a comparison of UQ strategies in ML-based predictions. This Python library unifies the benchmarking of multiple UQ metrics, including the calculation of nonstandard UQ metrics (combining information from the dataset and model), and provides a comprehensive evaluation. In this framework, UQ metrics are evaluated for different application scenarios, e.g., eliminating the predictions with the lowest confidence or obtaining a reliable uncertainty estimate for an acquisition function. Taken together, this library will help to standardize UQ investigations and evaluate new methodologies.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Novartis Biomedical Research, Novartis Campus, 4002 Basel, Switzerland.
ORCID-Profile: Nadine Schneider, Raquel Rodríguez-Pérez
Förderung
- Novartis
Zitieren
APA
Lanini, J., Huynh, M. T. D., Scebba, G., Schneider, N., & Rodríguez-Pérez, R. (2024). UNIQUE: A Framework for Uncertainty Quantification Benchmarking. Journal of Chemical Information and Modeling, 64(22), 8379-8386. https://doi.org/10.1021/acs.jcim.4c01578
Vancouver
Lanini J, Huynh MTD, Scebba G, Schneider N, Rodríguez-Pérez R. UNIQUE: A Framework for Uncertainty Quantification Benchmarking. Journal of Chemical Information and Modeling. 2024;64(22):8379-8386. doi:10.1021/acs.jcim.4c01578
BibTeX
@article{Lanini2024,
author = {Lanini, Jessica and Huynh, Minh Tam Davide and Scebba, Gaetano and Schneider, Nadine and Rodríguez-Pérez, Raquel},
title = {UNIQUE: A Framework for Uncertainty Quantification Benchmarking},
journal = {Journal of Chemical Information and Modeling},
year = {2024},
volume = {64},
number = {22},
pages = {8379--8386},
doi = {10.1021/acs.jcim.4c01578}
}
EUVIMED