EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Endowing protein language models with structural knowledge

Philip Hartout iD, Dexiong Chen, Paolo Pellizzoni, Carlos Oliver, Karsten Borgwardt iD

Bioinformatics · 2025

DOI: 10.1093/bioinformatics/btaf582 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics, 2025. Verfasst von Philip Hartout und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Machine Learning and Systems Biology, Max Planck Institute of Biochemistry, Bavaria, Martinsried 82152, Germany. Hauptthemen laut MEDLINE: Proteins, Computational Biology. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation Protein language models (PLMs) have transformed protein research by learning rich representations from sequence data alone, yet they largely ignore the wealth of structural information now available through advances in structure prediction. Current methods that incorporate structural data often require substantial computational resources and complex architectures, limiting their practical adoption. We present a novel joint sequence and structure embedding method that achieves computational and parameter efficiency while maintaining high performance. Our approach introduces a lightweight integration framework that combines pretrained sequence transformers’ self-attention with specialized structural adapters, enabling seamless incorporation of structural knowledge into existing PLMs through these enhanced self-attention mechanisms. Results The method demonstrates remarkable efficiency, requiring only modest pretraining on 542K protein structures, three orders of magnitude less than the data used to train PLMs, using standard masked language modeling objectives. Despite this lightweight approach, our joint embeddings consistently outperform sequence-only models like ESM-2 while achieving comparable results to more complex structure-based methods that use significantly more parameters and computational resources. This work establishes a new paradigm for protein representation learning that balances performance with practical constraints. By providing computationally efficient joint sequence-structure embeddings, we offer the scientific community an accessible tool that captures both sequential and structural protein information without the computational overhead typically associated with structure-aware models. Availability and implementation code and links to checkpoints are available at https://github.com/BorgwardtLab/PST.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ProteinsComputational BiologyProtein ConformationModels, MolecularSoftwareDatabases, ProteinAlgorithmsSequence Analysis, Protein

Substanzen

Proteins

Autor:innen und Institutionen

  • Department of Machine Learning and Systems Biology, Max Planck Institute of Biochemistry, Bavaria, Martinsried 82152, Germany.

ORCID-Profile: Philip Hartout, Karsten Borgwardt

Zitieren

APA

Hartout, P., Chen, D., Pellizzoni, P., Oliver, C., & Borgwardt, K. (2025). Endowing protein language models with structural knowledge. Bioinformatics. https://doi.org/10.1093/bioinformatics/btaf582

Vancouver

Hartout P, Chen D, Pellizzoni P, Oliver C, Borgwardt K. Endowing protein language models with structural knowledge. Bioinformatics. 2025. doi:10.1093/bioinformatics/btaf582

BibTeX
@article{Hartout2025,
  author = {Hartout, Philip and Chen, Dexiong and Pellizzoni, Paolo and Oliver, Carlos and Borgwardt, Karsten},
  title = {Endowing protein language models with structural knowledge},
  journal = {Bioinformatics},
  year = {2025},
  doi = {10.1093/bioinformatics/btaf582}
}
Kostenlos starten →