EUVIMEDEUVIMED entdecken
Crossref

Multi-Modal Modality-Masked Diffusion Network for Brain MRI Synthesis With Random Modality Missing

Xiangxi Meng, Kaicong Sun, Jun Xu, Xuming He, Dinggang Shen

IEEE Transactions on Medical Imaging · 2024-07 · 43 (7) · 2587-2598

DOI: 10.1109/tmi.2024.3368664 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2024-07. Verfasst von Xiangxi Meng und 4 weiteren Autor:innen. Hauptthemen laut MEDLINE: Brain, Algorithms. Die Arbeit wurde bisher 95-mal zitiert (Crossref) und nennt 52 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Science and Technology Commission of Shanghai Municipality.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Synthesis of unavailable imaging modalities from available ones can generate modality-specific complementary information and enable multi-modality based medical images diagnosis or treatment. Existing generative methods for medical image synthesis are usually based on cross-modal translation between acquired and missing modalities. These methods are usually dedicated to specific missing modality and perform synthesis in one shot, which cannot deal with varying number of missing modalities flexibly and construct the mapping across modalities effectively. To address the above issues, in this paper, we propose a unified Multi-modal Modality-masked Diffusion Network (M2DN), tackling multi-modal synthesis from the perspective of "progressive whole-modality inpainting", instead of "cross-modal translation". Specifically, our M2DN considers the missing modalities as random noise and takes all the modalities as a unity in each reverse diffusion step. The proposed joint synthesis scheme performs synthesis for the missing modalities and self-reconstruction for the available ones, which not only enables synthesis for arbitrary missing scenarios, but also facilitates the construction of common latent space and enhances the model representation ability. Besides, we introduce a modality-mask scheme to encode availability status of each incoming modality explicitly in a binary mask, which is adopted as condition for the diffusion model to further enhance the synthesis performance of our M2DN for arbitrary missing scenarios. We carry out experiments on two public brain MRI datasets for synthesis and downstream segmentation tasks. Experimental results demonstrate that our M2DN outperforms the state-of-the-art models significantly and shows great generalizability for arbitrary missing modalities.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

BrainAlgorithmsHumansImage Processing, Computer-AssistedMagnetic Resonance ImagingMultimodal Imaging

Förderung

  • National Natural Science Foundation of China
  • Science and Technology Commission of Shanghai Municipality

Zitieren

APA

Meng, X., Sun, K., Xu, J., He, X., & Shen, D. (2024). Multi-Modal Modality-Masked Diffusion Network for Brain MRI Synthesis With Random Modality Missing. IEEE Transactions on Medical Imaging, 43(7), 2587-2598. https://doi.org/10.1109/tmi.2024.3368664

Vancouver

Meng X, Sun K, Xu J, He X, Shen D. Multi-Modal Modality-Masked Diffusion Network for Brain MRI Synthesis With Random Modality Missing. IEEE Transactions on Medical Imaging. 2024;43(7):2587-2598. doi:10.1109/tmi.2024.3368664

BibTeX
@article{Meng2024,
  author = {Meng, Xiangxi and Sun, Kaicong and Xu, Jun and He, Xuming and Shen, Dinggang},
  title = {Multi-Modal Modality-Masked Diffusion Network for Brain MRI Synthesis With Random Modality Missing},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2024},
  volume = {43},
  number = {7},
  pages = {2587--2598},
  doi = {10.1109/tmi.2024.3368664}
}
Kostenlos starten →