Benchmarking uncertainty quantification for protein engineering
PLOS Computational Biology · 07.01.2025 · 21 (1) · e1012639
DOI: 10.1371/journal.pcbi.1012639 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Computational Biology, online erschienen am 07.01.2025. Verfasst von Kevin P Greenman und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Chemical Engineering, Catholic Institute of Technology, Cambridge, Massachusetts, United States of America. Hauptthemen laut MEDLINE: Benchmarking, Protein Engineering, Proteins. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Microsoft Research, National Science Foundation.
Abstract (Originalsprache)
Machine learning sequence-function models for proteins could enable significant advances in protein engineering, especially when paired with state-of-the-art methods to select new sequences for property optimization and/or model improvement. Such methods (Bayesian optimization and active learning) require calibrated estimations of model uncertainty. While studies have benchmarked a variety of deep learning uncertainty quantification (UQ) methods on standard and molecular machine-learning datasets, it is not clear if these results extend to protein datasets. In this work, we implemented a panel of deep learning UQ methods on regression tasks from the Fitness Landscape Inference for Proteins (FLIP) benchmark. We compared results across different degrees of distributional shift using metrics that assess each UQ method’s accuracy, calibration, coverage, width, and rank correlation. Additionally, we compared these metrics using one-hot encoding and pretrained language model representations, and we tested the UQ methods in retrospective active learning and Bayesian optimization settings. Our results indicate that there is no single best UQ method across all datasets, splits, and metrics, and that uncertainty-based sampling is often unable to outperform greedy sampling in Bayesian optimization. These benchmarks enable us to provide recommendations for more effective design of biological sequences using machine learning.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Substanzen
Autor:innen und Institutionen
- Department of Chemical Engineering, Catholic Institute of Technology, Cambridge, Massachusetts, United States of America.
- Department of Chemistry, Catholic Institute of Technology, Cambridge, Massachusetts, United States of America.
- Department of Chemical Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America.
- Microsoft Research, Cambridge, Massachusetts, United States of America.
ORCID-Profile: Kevin P Greenman, Ava P Amini, Kevin K Yang
Förderung
- Microsoft Research
- National Science Foundation
Zitieren
APA
Greenman, K. P., Amini, A. P., & Yang, K. K. (2025). Benchmarking uncertainty quantification for protein engineering. PLOS Computational Biology, 21(1), e1012639. https://doi.org/10.1371/journal.pcbi.1012639
Vancouver
Greenman KP, Amini AP, Yang KK. Benchmarking uncertainty quantification for protein engineering. PLOS Computational Biology. 2025;21(1):e1012639. doi:10.1371/journal.pcbi.1012639
BibTeX
@article{Greenman2025,
author = {Greenman, Kevin P and Amini, Ava P and Yang, Kevin K},
title = {Benchmarking uncertainty quantification for protein engineering},
journal = {PLOS Computational Biology},
year = {2025},
volume = {21},
number = {1},
pages = {e1012639},
doi = {10.1371/journal.pcbi.1012639}
}
EUVIMED