EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multimodal contrastive learning for spatial gene expression prediction using histology images

Wenwen Min iD, Zhiceng Shi, Jun Zhang, Jun Wan iD, Changmiao Wang iD

Briefings in Bioinformatics · 23.09.2024 · 25 (6)

DOI: 10.1093/bib/bbae551 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Briefings in Bioinformatics, online erschienen am 29.10.2024. Verfasst von Wenwen Min und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Information Science and Engineering, Yunnan University, East Outer Ring Road, Chenggong District, Kunming 650500, Yunnan, China. Hauptthemen laut MEDLINE: Gene Expression Profiling. Die Arbeit wurde bisher 100-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, the advent of spatial transcriptomics (ST) technology has unlocked unprecedented opportunities for delving into the complexities of gene expression patterns within intricate biological systems. Despite its transformative potential, the prohibitive cost of ST technology remains a significant barrier to its widespread adoption in large-scale studies. An alternative, more cost-effective strategy involves employing artificial intelligence to predict gene expression levels using readily accessible whole-slide images stained with Hematoxylin and Eosin (H&E). However, existing methods have yet to fully capitalize on multimodal information provided by H&E images and ST data with spatial location. In this paper, we propose mclSTExp, a multimodal contrastive learning with Transformer and Densenet-121 encoder for Spatial Transcriptomics Expression prediction. We conceptualize each spot as a “word”, integrating its intrinsic features with spatial context through the self-attention mechanism of a Transformer encoder. This integration is further enriched by incorporating image features via contrastive learning, thereby enhancing the predictive capability of our model. We conducted an extensive evaluation of highly variable genes in two breast cancer datasets and a skin squamous cell carcinoma dataset, and the results demonstrate that mclSTExp exhibits superior performance in predicting spatial gene expression. Moreover, mclSTExp has shown promise in interpreting cancer-specific overexpressed genes, elucidating immune-related genes, and identifying specialized spatial domains annotated by pathologists. Our source code is available at https://github.com/shizhiceng/mclSTExp.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Gene Expression ProfilingHumansBreast NeoplasmsAlgorithmsImage Processing, Computer-AssistedComputational BiologyTranscriptomeFemaleMachine LearningGene Expression Regulation, Neoplastic

Stichwörter der Autor:innen

histology imagesmultimodal contrastive learningspatial transcriptomicstransformer encoder

Autor:innen und Institutionen

  • School of Information Science and Engineering, Yunnan University, East Outer Ring Road, Chenggong District, Kunming 650500, Yunnan, China.
  • School of Information and Engineering, Zhongnan University of Economics and Law, 182 South Lake Avenue, East Lake New Technology Development Zone, Wuhan 430073, Hubei, China.
  • Medical Big Data, Shenzhen Research Institute of Big Data, Longxiang Boulevard, Longgang District, Shenzhen 518172, Guangdong, China.

ORCID-Profile: Wenwen Min, Jun Wan, Changmiao Wang

Zitieren

APA

Min, W., Shi, Z., Zhang, J., Wan, J., & Wang, C. (2024). Multimodal contrastive learning for spatial gene expression prediction using histology images. Briefings in Bioinformatics, 25(6). https://doi.org/10.1093/bib/bbae551

Vancouver

Min W, Shi Z, Zhang J, Wan J, Wang C. Multimodal contrastive learning for spatial gene expression prediction using histology images. Briefings in Bioinformatics. 2024;25(6). doi:10.1093/bib/bbae551

BibTeX
@article{Min2024,
  author = {Min, Wenwen and Shi, Zhiceng and Zhang, Jun and Wan, Jun and Wang, Changmiao},
  title = {Multimodal contrastive learning for spatial gene expression prediction using histology images},
  journal = {Briefings in Bioinformatics},
  year = {2024},
  volume = {25},
  number = {6},
  doi = {10.1093/bib/bbae551}
}
Kostenlos starten →