EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MULAN: multimodal protein language model for sequence and structure encoding

Daria Frolova iD, Marina Pak iD, Anna Litvin iD, Ilya Sharov iD, Dmitry Ivankov iD, Ivan Oseledets iD

Bioinformatics Advances · 26.12.2024 · 5 (1)

DOI: 10.1093/bioadv/vbaf117 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics Advances, online erschienen am 20.05.2025. Verfasst von Daria Frolova und 5 weiteren Autor:innen. Beteiligte Einrichtung: Ligand Pro, Moscow 121205, Russia. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation Most protein language models (PLMs) produce high-quality representations using only protein sequences. However, incorporating known protein structures is important for many prediction tasks, leading to increased interest in structure-aware PLMs. Currently, structure-aware PLMs are either trained from scratch or add significant parameter overhead for the structure encoder. Results In this study, we propose MULAN, a MULtimodal PLM for both sequence and ANgle-based structure encoding. MULAN has a pre-trained sequence encoder and an introduced parameter-efficient Structure Adapter, which are then fused and trained together. Based on the evaluation of nine downstream tasks, MULAN models of various sizes show a quality improvement compared to both sequence-only ESM2 and structure-aware SaProt. The main improvements are shown for the protein–protein interaction prediction (up to 0.12 in AUROC). Importantly, unlike other models, MULAN offers a cheap increase in structural awareness of protein representations because of the finetuning of existing PLMs instead of training from scratch. We perform a detailed analysis of the proposed model and demonstrate its awareness of the protein structure. Availability and implementation The implementation, training data, and model checkpoints are available at https://github.com/DFrolova/MULAN.

Autor:innen und Institutionen

  • Ligand Pro, Moscow 121205, Russia.
  • Center for Artificial Intelligence Technology, Skolkovo Institute of Science and Technology, Moscow 121205, Russia.
  • Center for Molecular and Cellular Biology, Skolkovo Institute of Science and Technology, Moscow 121205, Russia.
  • Belozersky Institute of Physico-Chemical Biology, Lomonosov Moscow State University, Moscow 119992, Russia.
  • AIRI, Moscow 105064, Russia.

ORCID-Profile: Daria Frolova, Marina Pak, Anna Litvin, Ilya Sharov, Dmitry Ivankov, Ivan Oseledets

Zitieren

APA

Frolova, D., Pak, M., Litvin, A., Sharov, I., Ivankov, D., & Oseledets, I. (2024). MULAN: multimodal protein language model for sequence and structure encoding. Bioinformatics Advances, 5(1). https://doi.org/10.1093/bioadv/vbaf117

Vancouver

Frolova D, Pak M, Litvin A, Sharov I, Ivankov D, Oseledets I. MULAN: multimodal protein language model for sequence and structure encoding. Bioinformatics Advances. 2024;5(1). doi:10.1093/bioadv/vbaf117

BibTeX
@article{Frolova2024,
  author = {Frolova, Daria and Pak, Marina and Litvin, Anna and Sharov, Ilya and Ivankov, Dmitry and Oseledets, Ivan},
  title = {MULAN: multimodal protein language model for sequence and structure encoding},
  journal = {Bioinformatics Advances},
  year = {2024},
  volume = {5},
  number = {1},
  doi = {10.1093/bioadv/vbaf117}
}
Kostenlos starten →