Trainable subnetworks reveal insights into structure knowledge organization in protein language models
PLOS Computational Biology · 2026
DOI: 10.1371/journal.pcbi.1013925 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Computational Biology, 2026. Verfasst von Ria Vinod und 3 weiteren Autor:innen. Beteiligte Einrichtung: Center for Computational and Molecular Biology, Brown University, Providence, Rhode Island, United States of America. Hauptthemen laut MEDLINE: Proteins. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Protein language models (PLMs) pretrained via a masked language modeling objective have proven effective across a range of structure-related tasks, including high-resolution structure prediction. However, it remains unclear to what extent these models factorize protein structural categories among their learned parameters. In this work, we introduce trainable subnetworks, which mask out the PLM weights responsible for language modeling performance on a structural category of proteins. We systematically trained 39 PLM subnetworks targeting both sequence- and residue-level features at varying degrees of resolution using annotations defined by the CATH taxonomy and secondary structure elements. Using these PLM subnetworks, we assessed how structural factorization in PLMs influences downstream structure prediction. Our results show that PLMs are highly sensitive to sequence-level features and can predominantly disentangle extremely coarse or fine-grained information. Furthermore, we observe that structure prediction is highly responsive to factorized PLM representations and that small changes in language modeling performance can significantly impair PLM-based structure prediction capabilities. Our work presents a framework for studying feature entanglement within pretrained PLMs and can be leveraged to improve the alignment of learned PLM representations with known biological concepts.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Substanzen
Autor:innen und Institutionen
- Center for Computational and Molecular Biology, Brown University, Providence, Rhode Island, United States of America.
- Microsoft Research, Cambridge, Massachusetts, United States of America.
ORCID-Profile: Kevin K Yang
Zitieren
APA
Vinod, R., Amini, A. P., Crawford, L., & Yang, K. K. (2026). Trainable subnetworks reveal insights into structure knowledge organization in protein language models. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1013925
Vancouver
Vinod R, Amini AP, Crawford L, Yang KK. Trainable subnetworks reveal insights into structure knowledge organization in protein language models. PLOS Computational Biology. 2026. doi:10.1371/journal.pcbi.1013925
BibTeX
@article{Vinod2026,
author = {Vinod, Ria and Amini, Ava P and Crawford, Lorin and Yang, Kevin K},
title = {Trainable subnetworks reveal insights into structure knowledge organization in protein language models},
journal = {PLOS Computational Biology},
year = {2026},
doi = {10.1371/journal.pcbi.1013925}
}
EUVIMED