EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Bilingual language model for protein sequence and structure

Michael Heinzinger iD, Konstantin Weissenow, Joaquin Gomez Sanchez, Adrian Henkel, Milot Mirdita, Martin Steinegger iD, Burkhard Rost

NAR Genomics and Bioinformatics · 28.09.2024 · 6 (4)

DOI: 10.1093/nargab/lqae150 ↗

Kurz gefasst

Fachartikel in der Zeitschrift NAR Genomics and Bioinformatics, online erschienen am 15.11.2024. Verfasst von Michael Heinzinger und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Computation, Information, and Technology (CIT), Department of Informatics, Bioinformatics & Computational Biology, TUM (Technical Universit…. Die Arbeit wurde bisher 281-mal zitiert (Crossref) und nennt 92 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Bavarian Ministry of Education, TUM, German Ministry for Research and Education.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Adapting language models to protein sequences spawned the development of powerful protein language models (pLMs). Concurrently, AlphaFold2 broke through in protein structure prediction. Now we can systematically and comprehensively explore the dual nature of proteins that act and exist as three-dimensional (3D) machines and evolve as linear strings of one-dimensional (1D) sequences. Here, we leverage pLMs to simultaneously model both modalities in a single model. We encode protein structures as token sequences using the 3Di-alphabet introduced by the 3D-alignment method Foldseek. For training, we built a non-redundant dataset from AlphaFoldDB and fine-tuned an existing pLM (ProtT5) to translate between 3Di and amino acid sequences. As a proof-of-concept for our novel approach, dubbed Protein ‘structure-sequence’ T5 (ProstT5), we showed improved performance for subsequent, structure-related prediction tasks, leading to three orders of magnitude speedup for deriving 3Di. This will be crucial for future applications trying to search metagenomic sequence databases at the sensitivity of structure comparisons. Our work showcased the potential of pLMs to tap into the information-rich protein structure revolution fueled by AlphaFold2. ProstT5 paves the way to develop new tools integrating the vast resource of 3D predictions and opens new research avenues in the post-AlphaFold2 era.

Autor:innen und Institutionen

  • School of Computation, Information, and Technology (CIT), Department of Informatics, Bioinformatics & Computational Biology, TUM (Technical University of Munich), 85748 Garching/Munich, Germany.
  • School of Biological Sciences, Seoul National University, 08826 Seoul, South Korea.
  • Artificial Intelligence Institute, Seoul National University, 08826 Seoul, South Korea.
  • Institute of Molecular Biology and Genetics, Seoul National University, 08826 Seoul, South Korea.
  • Institute for Advanced Study (TUM-IAS), Lichtenbergstr, 2a, 85748 Garching/Munich, Germany & TUM School of Life Sciences Weihenstephan (TUM-WZW), Alte Akademie 8, Freising, Germany.

ORCID-Profile: Michael Heinzinger, Martin Steinegger

Förderung

  • Bavarian Ministry of Education
  • TUM
  • German Ministry for Research and Education
  • Deutsche Forschungsgemeinschaft
  • National Research Foundation of Korea
  • Creative-Pioneering Researchers Program through Seoul National University

Zitieren

APA

Heinzinger, M., Weissenow, K., Sanchez, J. G., Henkel, A., Mirdita, M., Steinegger, M., & Rost, B. (2024). Bilingual language model for protein sequence and structure. NAR Genomics and Bioinformatics, 6(4). https://doi.org/10.1093/nargab/lqae150

Vancouver

Heinzinger M, Weissenow K, Sanchez JG, Henkel A, Mirdita M, Steinegger M, et al. Bilingual language model for protein sequence and structure. NAR Genomics and Bioinformatics. 2024;6(4). doi:10.1093/nargab/lqae150

BibTeX
@article{Heinzinger2024,
  author = {Heinzinger, Michael and Weissenow, Konstantin and Sanchez, Joaquin Gomez and Henkel, Adrian and Mirdita, Milot and Steinegger, Martin and Rost, Burkhard},
  title = {Bilingual language model for protein sequence and structure},
  journal = {NAR Genomics and Bioinformatics},
  year = {2024},
  volume = {6},
  number = {4},
  doi = {10.1093/nargab/lqae150}
}
Kostenlos starten →