MULAN: multimodal protein language model for sequence and structure encoding
Bioinformatics Advances · 26.12.2024 · 5 (1)
DOI: 10.1093/bioadv/vbaf117 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioinformatics Advances, online erschienen am 20.05.2025. Verfasst von Daria Frolova und 5 weiteren Autor:innen. Beteiligte Einrichtung: Ligand Pro, Moscow 121205, Russia. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Motivation Most protein language models (PLMs) produce high-quality representations using only protein sequences. However, incorporating known protein structures is important for many prediction tasks, leading to increased interest in structure-aware PLMs. Currently, structure-aware PLMs are either trained from scratch or add significant parameter overhead for the structure encoder. Results In this study, we propose MULAN, a MULtimodal PLM for both sequence and ANgle-based structure encoding. MULAN has a pre-trained sequence encoder and an introduced parameter-efficient Structure Adapter, which are then fused and trained together. Based on the evaluation of nine downstream tasks, MULAN models of various sizes show a quality improvement compared to both sequence-only ESM2 and structure-aware SaProt. The main improvements are shown for the protein–protein interaction prediction (up to 0.12 in AUROC). Importantly, unlike other models, MULAN offers a cheap increase in structural awareness of protein representations because of the finetuning of existing PLMs instead of training from scratch. We perform a detailed analysis of the proposed model and demonstrate its awareness of the protein structure. Availability and implementation The implementation, training data, and model checkpoints are available at https://github.com/DFrolova/MULAN.
Autor:innen und Institutionen
- Ligand Pro, Moscow 121205, Russia.
- Center for Artificial Intelligence Technology, Skolkovo Institute of Science and Technology, Moscow 121205, Russia.
- Center for Molecular and Cellular Biology, Skolkovo Institute of Science and Technology, Moscow 121205, Russia.
- Belozersky Institute of Physico-Chemical Biology, Lomonosov Moscow State University, Moscow 119992, Russia.
- AIRI, Moscow 105064, Russia.
ORCID-Profile: Daria Frolova, Marina Pak, Anna Litvin, Ilya Sharov, Dmitry Ivankov, Ivan Oseledets
Zitieren
APA
Frolova, D., Pak, M., Litvin, A., Sharov, I., Ivankov, D., & Oseledets, I. (2024). MULAN: multimodal protein language model for sequence and structure encoding. Bioinformatics Advances, 5(1). https://doi.org/10.1093/bioadv/vbaf117
Vancouver
Frolova D, Pak M, Litvin A, Sharov I, Ivankov D, Oseledets I. MULAN: multimodal protein language model for sequence and structure encoding. Bioinformatics Advances. 2024;5(1). doi:10.1093/bioadv/vbaf117
BibTeX
@article{Frolova2024,
author = {Frolova, Daria and Pak, Marina and Litvin, Anna and Sharov, Ilya and Ivankov, Dmitry and Oseledets, Ivan},
title = {MULAN: multimodal protein language model for sequence and structure encoding},
journal = {Bioinformatics Advances},
year = {2024},
volume = {5},
number = {1},
doi = {10.1093/bioadv/vbaf117}
}
EUVIMED