Tackling Dual-stage Missing Modalities in Brain Tumor Segmentation via Robust Modality Reconstruction and Prompt-guided Modality Adaptation
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (16) · 13314-13322
DOI: 10.1609/aaai.v40i16.38334 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yunpeng Zhao und 7 weiteren Autor:innen.
Abstract (Originalsprache)
Addressing missing modalities is a critical challenge in multimodal brain tumor segmentation. Most existing approaches merely handle modality-incomplete inputs during inference, assuming a full set of modalities for all training samples. However, this unrealistic assumption limits the usage of abundant modality-incomplete data commonly observed in clinical practice. In this paper, we explore a more practical task of tackling missing modalities during both training and inference. We propose a universal model featuring robust modality reconstruction and prompt-guided modality adaptation. Our mask-reconstruction pre-training enables robust modality-invariant representation learning, during which we design a novel distribution approximation method that supervises the reconstruction of absent modalities without requiring full-modal training data. Afterwards, when adapting our model to the segmentation task, we introduce the complete-then-distill (CTD) paradigm, which first estimates missing modalities in training samples from the available ones, and then distills the knowledge from the reconstructed full-modal representations to enhance learning from modality-incomplete data. Moreover, we propose prompt-guided modality adaptation to personalize a subset of model parameters during CTD, enabling the model to adapt to each distinct modality input scenario by using prompts with rich visual-textual information. Extensive experiments on two brain tumor segmentation benchmarks show our method consistently surpasses previous state-of-the-art approaches under dual-stage missing modality settings across various missing ratios.
Zitieren
APA
Zhao, Y., Chen, C., Pang, Q. Y., Fu, Y., Li, Q., Tang, C., Ang, B. T., & Jin, Y. (2026). Tackling Dual-stage Missing Modalities in Brain Tumor Segmentation via Robust Modality Reconstruction and Prompt-guided Modality Adaptation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(16), 13314-13322. https://doi.org/10.1609/aaai.v40i16.38334
Vancouver
Zhao Y, Chen C, Pang QY, Fu Y, Li Q, Tang C, et al. Tackling Dual-stage Missing Modalities in Brain Tumor Segmentation via Robust Modality Reconstruction and Prompt-guided Modality Adaptation. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(16):13314-13322. doi:10.1609/aaai.v40i16.38334
BibTeX
@article{Zhao2026,
author = {Zhao, Yunpeng and Chen, Cheng and Pang, Qing You and Fu, Yibing and Li, Quanzheng and Tang, Carol and Ang, Beng Ti and Jin, Yueming},
title = {Tackling Dual-stage Missing Modalities in Brain Tumor Segmentation via Robust Modality Reconstruction and Prompt-guided Modality Adaptation},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {16},
pages = {13314--13322},
doi = {10.1609/aaai.v40i16.38334}
}
EUVIMED