Multimodal contrastive learning for spatial gene expression prediction using histology images
Briefings in Bioinformatics · 23.09.2024 · 25 (6)
DOI: 10.1093/bib/bbae551 ↗Kurz gefasst
Fachartikel in der Zeitschrift Briefings in Bioinformatics, online erschienen am 29.10.2024. Verfasst von Wenwen Min und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Information Science and Engineering, Yunnan University, East Outer Ring Road, Chenggong District, Kunming 650500, Yunnan, China. Hauptthemen laut MEDLINE: Gene Expression Profiling. Die Arbeit wurde bisher 100-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In recent years, the advent of spatial transcriptomics (ST) technology has unlocked unprecedented opportunities for delving into the complexities of gene expression patterns within intricate biological systems. Despite its transformative potential, the prohibitive cost of ST technology remains a significant barrier to its widespread adoption in large-scale studies. An alternative, more cost-effective strategy involves employing artificial intelligence to predict gene expression levels using readily accessible whole-slide images stained with Hematoxylin and Eosin (H&E). However, existing methods have yet to fully capitalize on multimodal information provided by H&E images and ST data with spatial location. In this paper, we propose mclSTExp, a multimodal contrastive learning with Transformer and Densenet-121 encoder for Spatial Transcriptomics Expression prediction. We conceptualize each spot as a “word”, integrating its intrinsic features with spatial context through the self-attention mechanism of a Transformer encoder. This integration is further enriched by incorporating image features via contrastive learning, thereby enhancing the predictive capability of our model. We conducted an extensive evaluation of highly variable genes in two breast cancer datasets and a skin squamous cell carcinoma dataset, and the results demonstrate that mclSTExp exhibits superior performance in predicting spatial gene expression. Moreover, mclSTExp has shown promise in interpreting cancer-specific overexpressed genes, elucidating immune-related genes, and identifying specialized spatial domains annotated by pathologists. Our source code is available at https://github.com/shizhiceng/mclSTExp.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Information Science and Engineering, Yunnan University, East Outer Ring Road, Chenggong District, Kunming 650500, Yunnan, China.
- School of Information and Engineering, Zhongnan University of Economics and Law, 182 South Lake Avenue, East Lake New Technology Development Zone, Wuhan 430073, Hubei, China.
- Medical Big Data, Shenzhen Research Institute of Big Data, Longxiang Boulevard, Longgang District, Shenzhen 518172, Guangdong, China.
ORCID-Profile: Wenwen Min, Jun Wan, Changmiao Wang
Zitieren
APA
Min, W., Shi, Z., Zhang, J., Wan, J., & Wang, C. (2024). Multimodal contrastive learning for spatial gene expression prediction using histology images. Briefings in Bioinformatics, 25(6). https://doi.org/10.1093/bib/bbae551
Vancouver
Min W, Shi Z, Zhang J, Wan J, Wang C. Multimodal contrastive learning for spatial gene expression prediction using histology images. Briefings in Bioinformatics. 2024;25(6). doi:10.1093/bib/bbae551
BibTeX
@article{Min2024,
author = {Min, Wenwen and Shi, Zhiceng and Zhang, Jun and Wan, Jun and Wang, Changmiao},
title = {Multimodal contrastive learning for spatial gene expression prediction using histology images},
journal = {Briefings in Bioinformatics},
year = {2024},
volume = {25},
number = {6},
doi = {10.1093/bib/bbae551}
}
EUVIMED