EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Trainable subnetworks reveal insights into structure knowledge organization in protein language models

Ria Vinod, Ava P Amini, Lorin Crawford, Kevin K Yang iD

PLOS Computational Biology · 2026

DOI: 10.1371/journal.pcbi.1013925 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, 2026. Verfasst von Ria Vinod und 3 weiteren Autor:innen. Beteiligte Einrichtung: Center for Computational and Molecular Biology, Brown University, Providence, Rhode Island, United States of America. Hauptthemen laut MEDLINE: Proteins. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Protein language models (PLMs) pretrained via a masked language modeling objective have proven effective across a range of structure-related tasks, including high-resolution structure prediction. However, it remains unclear to what extent these models factorize protein structural categories among their learned parameters. In this work, we introduce trainable subnetworks, which mask out the PLM weights responsible for language modeling performance on a structural category of proteins. We systematically trained 39 PLM subnetworks targeting both sequence- and residue-level features at varying degrees of resolution using annotations defined by the CATH taxonomy and secondary structure elements. Using these PLM subnetworks, we assessed how structural factorization in PLMs influences downstream structure prediction. Our results show that PLMs are highly sensitive to sequence-level features and can predominantly disentangle extremely coarse or fine-grained information. Furthermore, we observe that structure prediction is highly responsive to factorized PLM representations and that small changes in language modeling performance can significantly impair PLM-based structure prediction capabilities. Our work presents a framework for studying feature entanglement within pretrained PLMs and can be leveraged to improve the alignment of learned PLM representations with known biological concepts.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ProteinsComputational BiologyProtein ConformationModels, MolecularDatabases, ProteinLarge Language Models

Substanzen

Proteins

Autor:innen und Institutionen

  • Center for Computational and Molecular Biology, Brown University, Providence, Rhode Island, United States of America.
  • Microsoft Research, Cambridge, Massachusetts, United States of America.

ORCID-Profile: Kevin K Yang

Zitieren

APA

Vinod, R., Amini, A. P., Crawford, L., & Yang, K. K. (2026). Trainable subnetworks reveal insights into structure knowledge organization in protein language models. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1013925

Vancouver

Vinod R, Amini AP, Crawford L, Yang KK. Trainable subnetworks reveal insights into structure knowledge organization in protein language models. PLOS Computational Biology. 2026. doi:10.1371/journal.pcbi.1013925

BibTeX
@article{Vinod2026,
  author = {Vinod, Ria and Amini, Ava P and Crawford, Lorin and Yang, Kevin K},
  title = {Trainable subnetworks reveal insights into structure knowledge organization in protein language models},
  journal = {PLOS Computational Biology},
  year = {2026},
  doi = {10.1371/journal.pcbi.1013925}
}
Kostenlos starten →