EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

A deep learning representation and spatial Bayesian cell-type deconvolution for spatial transcriptomics

Xiao Yang, Yanbin Feng, Yanfang Zhao, Huamei Li, Xiaozhou Chen

PeerJ · 23.07.2026 · 14 · e21548

DOI: 10.7717/peerj.21548 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PeerJ, online erschienen am 23.07.2026. Verfasst von Xiao Yang und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Mathematics and Computer Science, Yunnan Minzu University, Kunming, Yunnan, China. Hauptthemen laut MEDLINE: Spatial Transcriptomics, Deep Learning, Carcinoma, Pancreatic Ductal, Transcriptome. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Natural Science Foundation of China, Yunnan Province Key Laboratory of Digital-Intelligent Ethnic Computing.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Spatial transcriptomics provides unprecedented insights into gene expression in the spatial context of tissues; however, some mainstream techniques lack quantitative analysis of uncertainty.

Methods

In this study, we propose a computational framework—Spatial Deconvolution via Deep Gaussian Processes (SDDGP)—that integrates deep Gaussian processes with Bayesian inference for spatially aware deconvolution of spatial transcriptomics data. The framework consists of three tightly coupled components. A multi-layer deep Gaussian process (DGP) captures nonlinear spatial dependencies in cell-type composition by propagating spatial coordinates through successive Gaussian process (GP) layers, each equipped with sparse variational inducing points and a radial basis function (RBF) kernel that can be combined with an external Matern spatial kernel. The DGP output serves as an adaptive, spatially informed prior for per-spot cell-type proportions, which are then estimated via a Negative-Binomial Markov chain Monte Carlo (MCMC) sampler with automatic proposal tuning, thinning, and split-chain convergence diagnostics.

Results

We compared our method with existing approaches on two datasets: a pancreatic ductal adenocarcinoma (PDAC) dataset with matched single-cell RNA sequencing and spatial transcriptomics, and a human thymus dataset. Across both datasets, our method estimated cell type proportions more accurately than existing methods, particularly for rare cell types, and recovered biologically meaningful spatial patterns such as the distribution of T cells and plasma cells in PDAC. The Bayesian formulation also provides a quantification of uncertainty for each estimate, which aids the interpretation of spatial heterogeneity in complex tissues.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Spatial TranscriptomicsDeep LearningCarcinoma, Pancreatic DuctalTranscriptomeBayes TheoremHumansMonte Carlo MethodPancreatic Neoplasms

Stichwörter der Autor:innen

DeconvolutionDeep Gaussian processesMarkov chain Monte CarloSpatial transcriptomics

Autor:innen und Institutionen

  • School of Mathematics and Computer Science, Yunnan Minzu University, Kunming, Yunnan, China.
  • Department of Rheumatology and Immunology, Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, Jiangsu, China.

Förderung

  • National Natural Science Foundation of China
  • Yunnan Province Key Laboratory of Digital-Intelligent Ethnic Computing

Zitieren

APA

Yang, X., Feng, Y., Zhao, Y., Li, H., & Chen, X. (2026). A deep learning representation and spatial Bayesian cell-type deconvolution for spatial transcriptomics. PeerJ, 14, e21548. https://doi.org/10.7717/peerj.21548

Vancouver

Yang X, Feng Y, Zhao Y, Li H, Chen X. A deep learning representation and spatial Bayesian cell-type deconvolution for spatial transcriptomics. PeerJ. 2026;14:e21548. doi:10.7717/peerj.21548

BibTeX
@article{Yang2026,
  author = {Yang, Xiao and Feng, Yanbin and Zhao, Yanfang and Li, Huamei and Chen, Xiaozhou},
  title = {A deep learning representation and spatial Bayesian cell-type deconvolution for spatial transcriptomics},
  journal = {PeerJ},
  year = {2026},
  volume = {14},
  pages = {e21548},
  doi = {10.7717/peerj.21548}
}
Kostenlos starten →