Endowing protein language models with structural knowledge
Bioinformatics · 2025
DOI: 10.1093/bioinformatics/btaf582 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioinformatics, 2025. Verfasst von Philip Hartout und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Machine Learning and Systems Biology, Max Planck Institute of Biochemistry, Bavaria, Martinsried 82152, Germany. Hauptthemen laut MEDLINE: Proteins, Computational Biology. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Motivation Protein language models (PLMs) have transformed protein research by learning rich representations from sequence data alone, yet they largely ignore the wealth of structural information now available through advances in structure prediction. Current methods that incorporate structural data often require substantial computational resources and complex architectures, limiting their practical adoption. We present a novel joint sequence and structure embedding method that achieves computational and parameter efficiency while maintaining high performance. Our approach introduces a lightweight integration framework that combines pretrained sequence transformers’ self-attention with specialized structural adapters, enabling seamless incorporation of structural knowledge into existing PLMs through these enhanced self-attention mechanisms. Results The method demonstrates remarkable efficiency, requiring only modest pretraining on 542K protein structures, three orders of magnitude less than the data used to train PLMs, using standard masked language modeling objectives. Despite this lightweight approach, our joint embeddings consistently outperform sequence-only models like ESM-2 while achieving comparable results to more complex structure-based methods that use significantly more parameters and computational resources. This work establishes a new paradigm for protein representation learning that balances performance with practical constraints. By providing computationally efficient joint sequence-structure embeddings, we offer the scientific community an accessible tool that captures both sequential and structural protein information without the computational overhead typically associated with structure-aware models. Availability and implementation code and links to checkpoints are available at https://github.com/BorgwardtLab/PST.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Substanzen
Autor:innen und Institutionen
- Department of Machine Learning and Systems Biology, Max Planck Institute of Biochemistry, Bavaria, Martinsried 82152, Germany.
ORCID-Profile: Philip Hartout, Karsten Borgwardt
Zitieren
APA
Hartout, P., Chen, D., Pellizzoni, P., Oliver, C., & Borgwardt, K. (2025). Endowing protein language models with structural knowledge. Bioinformatics. https://doi.org/10.1093/bioinformatics/btaf582
Vancouver
Hartout P, Chen D, Pellizzoni P, Oliver C, Borgwardt K. Endowing protein language models with structural knowledge. Bioinformatics. 2025. doi:10.1093/bioinformatics/btaf582
BibTeX
@article{Hartout2025,
author = {Hartout, Philip and Chen, Dexiong and Pellizzoni, Paolo and Oliver, Carlos and Borgwardt, Karsten},
title = {Endowing protein language models with structural knowledge},
journal = {Bioinformatics},
year = {2025},
doi = {10.1093/bioinformatics/btaf582}
}
EUVIMED