Bilingual language model for protein sequence and structure
NAR Genomics and Bioinformatics · 28.09.2024 · 6 (4)
DOI: 10.1093/nargab/lqae150 ↗Kurz gefasst
Fachartikel in der Zeitschrift NAR Genomics and Bioinformatics, online erschienen am 15.11.2024. Verfasst von Michael Heinzinger und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Computation, Information, and Technology (CIT), Department of Informatics, Bioinformatics & Computational Biology, TUM (Technical Universit…. Die Arbeit wurde bisher 281-mal zitiert (Crossref) und nennt 92 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Bavarian Ministry of Education, TUM, German Ministry for Research and Education.
Abstract (Originalsprache)
Adapting language models to protein sequences spawned the development of powerful protein language models (pLMs). Concurrently, AlphaFold2 broke through in protein structure prediction. Now we can systematically and comprehensively explore the dual nature of proteins that act and exist as three-dimensional (3D) machines and evolve as linear strings of one-dimensional (1D) sequences. Here, we leverage pLMs to simultaneously model both modalities in a single model. We encode protein structures as token sequences using the 3Di-alphabet introduced by the 3D-alignment method Foldseek. For training, we built a non-redundant dataset from AlphaFoldDB and fine-tuned an existing pLM (ProtT5) to translate between 3Di and amino acid sequences. As a proof-of-concept for our novel approach, dubbed Protein ‘structure-sequence’ T5 (ProstT5), we showed improved performance for subsequent, structure-related prediction tasks, leading to three orders of magnitude speedup for deriving 3Di. This will be crucial for future applications trying to search metagenomic sequence databases at the sensitivity of structure comparisons. Our work showcased the potential of pLMs to tap into the information-rich protein structure revolution fueled by AlphaFold2. ProstT5 paves the way to develop new tools integrating the vast resource of 3D predictions and opens new research avenues in the post-AlphaFold2 era.
Autor:innen und Institutionen
- School of Computation, Information, and Technology (CIT), Department of Informatics, Bioinformatics & Computational Biology, TUM (Technical University of Munich), 85748 Garching/Munich, Germany.
- School of Biological Sciences, Seoul National University, 08826 Seoul, South Korea.
- Artificial Intelligence Institute, Seoul National University, 08826 Seoul, South Korea.
- Institute of Molecular Biology and Genetics, Seoul National University, 08826 Seoul, South Korea.
- Institute for Advanced Study (TUM-IAS), Lichtenbergstr, 2a, 85748 Garching/Munich, Germany & TUM School of Life Sciences Weihenstephan (TUM-WZW), Alte Akademie 8, Freising, Germany.
ORCID-Profile: Michael Heinzinger, Martin Steinegger
Förderung
- Bavarian Ministry of Education
- TUM
- German Ministry for Research and Education
- Deutsche Forschungsgemeinschaft
- National Research Foundation of Korea
- Creative-Pioneering Researchers Program through Seoul National University
Zitieren
APA
Heinzinger, M., Weissenow, K., Sanchez, J. G., Henkel, A., Mirdita, M., Steinegger, M., & Rost, B. (2024). Bilingual language model for protein sequence and structure. NAR Genomics and Bioinformatics, 6(4). https://doi.org/10.1093/nargab/lqae150
Vancouver
Heinzinger M, Weissenow K, Sanchez JG, Henkel A, Mirdita M, Steinegger M, et al. Bilingual language model for protein sequence and structure. NAR Genomics and Bioinformatics. 2024;6(4). doi:10.1093/nargab/lqae150
BibTeX
@article{Heinzinger2024,
author = {Heinzinger, Michael and Weissenow, Konstantin and Sanchez, Joaquin Gomez and Henkel, Adrian and Mirdita, Milot and Steinegger, Martin and Rost, Burkhard},
title = {Bilingual language model for protein sequence and structure},
journal = {NAR Genomics and Bioinformatics},
year = {2024},
volume = {6},
number = {4},
doi = {10.1093/nargab/lqae150}
}
EUVIMED