EUVIMEDEUVIMED entdecken
Crossref

ProGen2: Exploring the boundaries of protein language models

Erik Nijkamp, Jeffrey A Ruffolo, Eli N Weinstein, Nikhil Naik, Ali Madani

Cell Systems · 2023-11 · 14 (11) · 968-978.e3

DOI: 10.1016/j.cels.2023.10.002 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cell Systems, 2023-11. Verfasst von Erik Nijkamp und 4 weiteren Autor:innen. Beteiligte Einrichtung: Salesforce Research, Palo Alto, CA, USA. Hauptthemen laut MEDLINE: Proteins, Artificial Intelligence. Die Arbeit wurde bisher 493-mal zitiert (Crossref) und nennt 59 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Attention-based models trained on protein sequences have demonstrated incredible success at classification and generation tasks relevant for artificial-intelligence-driven protein design. However, we lack a sufficient understanding of how very large-scale models and data play a role in effective protein model development. We introduce a suite of protein language models, named ProGen2, that are scaled up to 6.4B parameters and trained on different sequence datasets drawn from over a billion proteins from genomic, metagenomic, and immune repertoire databases. ProGen2 models show state-of-the-art performance in capturing the distribution of observed evolutionary sequences, generating novel viable sequences, and predicting protein fitness without additional fine-tuning. As large model sizes and raw numbers of protein sequences continue to become more widely accessible, our results suggest that a growing emphasis needs to be placed on the data distribution provided to a protein sequence model. Our models and code are open sourced for widespread adoption in protein engineering. A record of this paper's Transparent Peer Review process is included in the supplemental information.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ProteinsArtificial IntelligenceAmino Acid SequenceLanguageDatabases, Factual

Stichwörter der Autor:innen

fitness predictionlanguage modelingprotein design

Substanzen

Proteins

Autor:innen und Institutionen

  • Salesforce Research, Palo Alto, CA, USA.
  • Program in Molecular Biophysics, The Johns Hopkins University, Baltimore, MD, USA; Profluent Bio, Berkeley, CA, USA.
  • Data Science Institute, Columbia University, New York, NY, USA.
  • Salesforce Research, Palo Alto, CA, USA; Profluent Bio, Berkeley, CA, USA. Electronic address: ali@profluent.bio.

Zitieren

APA

Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., & Madani, A. (2023). ProGen2: Exploring the boundaries of protein language models. Cell Systems, 14(11), 968-978.e3. https://doi.org/10.1016/j.cels.2023.10.002

Vancouver

Nijkamp E, Ruffolo JA, Weinstein EN, Naik N, Madani A. ProGen2: Exploring the boundaries of protein language models. Cell Systems. 2023;14(11):968-978.e3. doi:10.1016/j.cels.2023.10.002

BibTeX
@article{Nijkamp2023,
  author = {Nijkamp, Erik and Ruffolo, Jeffrey A and Weinstein, Eli N and Naik, Nikhil and Madani, Ali},
  title = {ProGen2: Exploring the boundaries of protein language models},
  journal = {Cell Systems},
  year = {2023},
  volume = {14},
  number = {11},
  pages = {968--978.e3},
  doi = {10.1016/j.cels.2023.10.002}
}
Kostenlos starten →