Multimodal Co-Attention Fusion Network With Online Data Augmentation for Cancer Subtype Classification
IEEE Transactions on Medical Imaging · 2024-11 · 43 (11) · 3977-3989
DOI: 10.1109/tmi.2024.3405535 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2024-11. Verfasst von Saisai Ding und 4 weiteren Autor:innen. Hauptthemen laut MEDLINE: Neoplasms, Algorithms. Die Arbeit wurde bisher 31-mal zitiert (Crossref) und nennt 42 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, 111 Project.
Abstract (Originalsprache)
It is an essential task to accurately diagnose cancer subtypes in computational pathology for personalized cancer treatment. Recent studies have indicated that the combination of multimodal data, such as whole slide images (WSIs) and multi-omics data, could achieve more accurate diagnosis. However, robust cancer diagnosis remains challenging due to the heterogeneity among multimodal data, as well as the performance degradation caused by insufficient multimodal patient data. In this work, we propose a novel multimodal co-attention fusion network (MCFN) with online data augmentation (ODA) for cancer subtype classification. Specifically, a multimodal mutual-guided co-attention (MMC) module is proposed to effectively perform dense multimodal interactions. It enables multimodal data to mutually guide and calibrate each other during the integration process to alleviate inter- and intra-modal heterogeneities. Subsequently, a self-normalizing network (SNN)-Mixer is developed to allow information communication among different omics data and alleviate the high-dimensional small-sample size problem in multi-omics data. Most importantly, to compensate for insufficient multimodal samples for model training, we propose an ODA module in MCFN. The ODA module leverages the multimodal knowledge to guide the data augmentations of WSIs and maximize the data diversity during model training. Extensive experiments are conducted on the public TCGA dataset. The experimental results demonstrate that the proposed MCFN outperforms all the compared algorithms, suggesting its effectiveness.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- 111 Project
Zitieren
APA
Ding, S., Li, J., Wang, J., Ying, S., & Shi, J. (2024). Multimodal Co-Attention Fusion Network With Online Data Augmentation for Cancer Subtype Classification. IEEE Transactions on Medical Imaging, 43(11), 3977-3989. https://doi.org/10.1109/tmi.2024.3405535
Vancouver
Ding S, Li J, Wang J, Ying S, Shi J. Multimodal Co-Attention Fusion Network With Online Data Augmentation for Cancer Subtype Classification. IEEE Transactions on Medical Imaging. 2024;43(11):3977-3989. doi:10.1109/tmi.2024.3405535
BibTeX
@article{Ding2024,
author = {Ding, Saisai and Li, Juncheng and Wang, Jun and Ying, Shihui and Shi, Jun},
title = {Multimodal Co-Attention Fusion Network With Online Data Augmentation for Cancer Subtype Classification},
journal = {IEEE Transactions on Medical Imaging},
year = {2024},
volume = {43},
number = {11},
pages = {3977--3989},
doi = {10.1109/tmi.2024.3405535}
}
EUVIMED