ProGen2: Exploring the boundaries of protein language models
Cell Systems · 2023-11 · 14 (11) · 968-978.e3
DOI: 10.1016/j.cels.2023.10.002 ↗Kurz gefasst
Fachartikel in der Zeitschrift Cell Systems, 2023-11. Verfasst von Erik Nijkamp und 4 weiteren Autor:innen. Beteiligte Einrichtung: Salesforce Research, Palo Alto, CA, USA. Hauptthemen laut MEDLINE: Proteins, Artificial Intelligence. Die Arbeit wurde bisher 493-mal zitiert (Crossref) und nennt 59 Literaturangaben.
Abstract (Originalsprache)
Attention-based models trained on protein sequences have demonstrated incredible success at classification and generation tasks relevant for artificial-intelligence-driven protein design. However, we lack a sufficient understanding of how very large-scale models and data play a role in effective protein model development. We introduce a suite of protein language models, named ProGen2, that are scaled up to 6.4B parameters and trained on different sequence datasets drawn from over a billion proteins from genomic, metagenomic, and immune repertoire databases. ProGen2 models show state-of-the-art performance in capturing the distribution of observed evolutionary sequences, generating novel viable sequences, and predicting protein fitness without additional fine-tuning. As large model sizes and raw numbers of protein sequences continue to become more widely accessible, our results suggest that a growing emphasis needs to be placed on the data distribution provided to a protein sequence model. Our models and code are open sourced for widespread adoption in protein engineering. A record of this paper's Transparent Peer Review process is included in the supplemental information.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Substanzen
Autor:innen und Institutionen
- Salesforce Research, Palo Alto, CA, USA.
- Program in Molecular Biophysics, The Johns Hopkins University, Baltimore, MD, USA; Profluent Bio, Berkeley, CA, USA.
- Data Science Institute, Columbia University, New York, NY, USA.
- Salesforce Research, Palo Alto, CA, USA; Profluent Bio, Berkeley, CA, USA. Electronic address: ali@profluent.bio.
Zitieren
APA
Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., & Madani, A. (2023). ProGen2: Exploring the boundaries of protein language models. Cell Systems, 14(11), 968-978.e3. https://doi.org/10.1016/j.cels.2023.10.002
Vancouver
Nijkamp E, Ruffolo JA, Weinstein EN, Naik N, Madani A. ProGen2: Exploring the boundaries of protein language models. Cell Systems. 2023;14(11):968-978.e3. doi:10.1016/j.cels.2023.10.002
BibTeX
@article{Nijkamp2023,
author = {Nijkamp, Erik and Ruffolo, Jeffrey A and Weinstein, Eli N and Naik, Nikhil and Madani, Ali},
title = {ProGen2: Exploring the boundaries of protein language models},
journal = {Cell Systems},
year = {2023},
volume = {14},
number = {11},
pages = {968--978.e3},
doi = {10.1016/j.cels.2023.10.002}
}
EUVIMED