CD-Former: A Cross-Modal Dual-Interaction Transformer With Whole-Slide Image Pyramids and Genomics for Survival Prediction
IEEE Transactions on Circuits and Systems for Video Technology · 2026-07 · 36 (7) · 10868-10881
DOI: 10.1109/tcsvt.2026.3672108 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-07. Verfasst von Lifan Long und 6 weiteren Autor:innen. Die Arbeit nennt 63 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Sichuan Science and Technology Program, Key Laboratory of Internet Natural Language Processing of Sichuan Provincial Education Department.
Abstract (Originalsprache)
Survival prediction is crucial for cancer patients as it provides essential early prognostic information for treatment planning and decision making. Despite impressive performance, current multi-modal survival prediction methods that integrate pathology and genomic data face two main challenges: 1) Whole-slide images (WSIs) generally exhibit hierarchical structures, but the interactions of phenotypes at different resolutions remain unexplored. More importantly, the potential semantic discrepancy arising from diverse resolutions is often ignored. 2) The absence of effective interactions between the inherent hierarchical structures of WSIs and genomic data. To address these challenges, in this paper, we propose Cross-modal Dual-interaction Transformer (CD-Former), a robust hierarchical framework for multi-modal survival prediction. Our CD-Former involves two key components: 1) an Multimodal Cross-Scale Calibration (MCSC) module for effectively capturing correlations across multiple resolutions and calibrating fine-grained features, thereby bridging the semantic discrepancy caused by different WSI resolutions; and 2) a hierarchical interaction module termed Multi-modal Dual-interaction (M2Di) for fully exploring multi-resolution cross-modal correlations and interactions, which comprises a Patch-level Cross-Attention Block (PCAB) and a Region-level Cross-Attention Block (RCAB) to investigate cross-modal associations between patch- or region-level features of WSI and genomic data. Additionally, we employ a scale-oriented WSI enhancer to capture the interactions among various components of WSIs. The experimental results demonstrate the effectiveness of our proposed framework, which achieves state-of-the-art performance compared to previous studies.
Förderung
- National Natural Science Foundation of China
- Sichuan Science and Technology Program
- Key Laboratory of Internet Natural Language Processing of Sichuan Provincial Education Department
Zitieren
APA
Long, L., Peng, X., Cui, J., Liu, B., Wu, X., Zhang, D., & Wang, Y. (2026). CD-Former: A Cross-Modal Dual-Interaction Transformer With Whole-Slide Image Pyramids and Genomics for Survival Prediction. IEEE Transactions on Circuits and Systems for Video Technology, 36(7), 10868-10881. https://doi.org/10.1109/tcsvt.2026.3672108
Vancouver
Long L, Peng X, Cui J, Liu B, Wu X, Zhang D, et al. CD-Former: A Cross-Modal Dual-Interaction Transformer With Whole-Slide Image Pyramids and Genomics for Survival Prediction. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(7):10868-10881. doi:10.1109/tcsvt.2026.3672108
BibTeX
@article{Long2026,
author = {Long, Lifan and Peng, Xingchen and Cui, Jiaqi and Liu, Bo and Wu, Xi and Zhang, Daoqiang and Wang, Yan},
title = {CD-Former: A Cross-Modal Dual-Interaction Transformer With Whole-Slide Image Pyramids and Genomics for Survival Prediction},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {7},
pages = {10868--10881},
doi = {10.1109/tcsvt.2026.3672108}
}
EUVIMED