EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Paying attention to attention: High attention sites as indicators of protein family and function in language models

Gowri Nayar iD, Alp Tartici, Russ B Altman iD

PLOS Computational Biology · 12.09.2025 · 21 (9) · e1013424

DOI: 10.1371/journal.pcbi.1013424 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, online erschienen am 12.09.2025. Verfasst von Gowri Nayar und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Data Science, Stanford University, Stanford, California, United States of America. Hauptthemen laut MEDLINE: Proteins. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NLM NIH HHS, NIGMS NIH HHS, U.S. National Library of Medicine.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Protein Language Models (PLMs) use transformer architectures to capture patterns within protein primary sequences, providing a powerful computational representation of the amino acid sequence. Through large-scale training on protein primary sequences, PLMs generate vector representations that encapsulate the biochemical and structural properties of proteins. At the core of PLMs is the attention mechanism, which facilitates the capture of long-range dependencies by computing pairwise importance scores across residues, thereby highlighting regions of biological interaction within the sequence. The attention matrices offer an untapped opportunity to uncover specific biological properties of proteins, particularly their functions. In this work, we introduce a novel approach, using the Evolutionary Scale Modelling (ESM), for identifying High Attention (HA) sites within protein primary sequences, corresponding to key residues that define protein families. By examining attention patterns across multiple layers, we pinpoint residues that contribute most to family classification and function prediction. Our contributions are as follows: (1) we propose a method for identifying HA sites at critical residues from the middle layers of the PLM; (2) we demonstrate that these HA sites provide interpretable links to biological functions; and (3) we show that HA sites improve active site predictions for functions of unannotated proteins. We make available the HA sites for the human proteome. This work offers a broadly applicable approach to protein classification and functional annotation and provides a biological interpretation of the PLM’s representation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ProteinsComputational BiologyAmino Acid SequenceHumansSequence Analysis, ProteinAlgorithmsDatabases, Protein

Substanzen

Proteins

Autor:innen und Institutionen

  • Department of Biomedical Data Science, Stanford University, Stanford, California, United States of America.
  • Department of Genetics, Stanford University, Stanford, California, United States of America.
  • Department of Medicine, Stanford University, Stanford, California, United States of America.
  • Department of Bioengineering, Stanford University, Stanford, California, United States of America.

ORCID-Profile: Gowri Nayar, Russ B Altman

Förderung

  • NLM NIH HHS
  • NIGMS NIH HHS
  • U.S. National Library of Medicine
  • Smith Fellowship
  • National Institute of General Medical Sciences
  • Stanford Institute for Human-Centered Artificial Intelligence, Stanford University
  • Chan Zuckerberg Initiative

Zitieren

APA

Nayar, G., Tartici, A., & Altman, R. B. (2025). Paying attention to attention: High attention sites as indicators of protein family and function in language models. PLOS Computational Biology, 21(9), e1013424. https://doi.org/10.1371/journal.pcbi.1013424

Vancouver

Nayar G, Tartici A, Altman RB. Paying attention to attention: High attention sites as indicators of protein family and function in language models. PLOS Computational Biology. 2025;21(9):e1013424. doi:10.1371/journal.pcbi.1013424

BibTeX
@article{Nayar2025,
  author = {Nayar, Gowri and Tartici, Alp and Altman, Russ B},
  title = {Paying attention to attention: High attention sites as indicators of protein family and function in language models},
  journal = {PLOS Computational Biology},
  year = {2025},
  volume = {21},
  number = {9},
  pages = {e1013424},
  doi = {10.1371/journal.pcbi.1013424}
}
Kostenlos starten →