EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Aggregating residue-level protein language model embeddings with optimal transport

Navid NaderiAlizadeh iD, Rohit Singh iD

Bioinformatics Advances · 26.12.2024 · 5 (1)

DOI: 10.1093/bioadv/vbaf060 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics Advances, online erschienen am 20.03.2025. Verfasst von Navid NaderiAlizadeh und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics and Bioinformatics, Duke University, Durham, NC 27705, United States. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Whitehead Scholarship.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation Protein language models (PLMs) have emerged as powerful approaches for mapping protein sequences into embeddings suitable for various applications. As protein representation schemes, PLMs generate per-token (i.e. per-residue) representations, resulting in variable-sized outputs based on protein length. This variability poses a challenge for protein-level prediction tasks that require uniform-sized embeddings for consistent analysis across different proteins. Previous work has typically used average pooling to summarize token-level PLM outputs, but it is unclear whether this method effectively prioritizes the relevant information across token-level representations. Results We introduce a novel method utilizing optimal transport to convert variable-length PLM outputs into fixed-length representations. We conceptualize per-token PLM outputs as samples from a probabilistic distribution and employ sliced-Wasserstein distances to map these samples against a reference set, creating a Euclidean embedding in the output space. The resulting embedding is agnostic to the length of the input and represents the entire protein. We demonstrate the superiority of our method over average pooling for several downstream prediction tasks, particularly with constrained PLM sizes, enabling smaller-scale PLMs to match or exceed the performance of average-pooled larger-scale PLMs. Our aggregation scheme is especially effective for longer protein sequences by capturing essential information that might be lost through average pooling. Availability and implementation Our implementation code can be found at https://github.com/navid-naderi/PLM_SWE.

Autor:innen und Institutionen

  • Department of Biostatistics and Bioinformatics, Duke University, Durham, NC 27705, United States.
  • Department of Cell Biology, Duke University, Durham, NC 27705, United States.

ORCID-Profile: Navid NaderiAlizadeh, Rohit Singh

Förderung

  • Whitehead Scholarship

Zitieren

APA

NaderiAlizadeh, N., & Singh, R. (2024). Aggregating residue-level protein language model embeddings with optimal transport. Bioinformatics Advances, 5(1). https://doi.org/10.1093/bioadv/vbaf060

Vancouver

NaderiAlizadeh N, Singh R. Aggregating residue-level protein language model embeddings with optimal transport. Bioinformatics Advances. 2024;5(1). doi:10.1093/bioadv/vbaf060

BibTeX
@article{NaderiAlizadeh2024,
  author = {NaderiAlizadeh, Navid and Singh, Rohit},
  title = {Aggregating residue-level protein language model embeddings with optimal transport},
  journal = {Bioinformatics Advances},
  year = {2024},
  volume = {5},
  number = {1},
  doi = {10.1093/bioadv/vbaf060}
}
Kostenlos starten →