EUVIMEDSkopri EUVIMED
Crossref

A 3D hierarchical cross‐modality interaction network using transformers and convolutions for brain glioma segmentation in MR images

Yuzhou Zhuang, Hong Liu, Wei Fang, Guangzhi Ma, Sisi Sun, Yunfeng Zhu, Xu Zhang, Chuanbin Ge, Wenyang Chen, Jiaosong Long, Enmin Song

Medical Physics · 2024

DOI: 10.1002/mp.17354 ↗

At a glance

Journal article in Medical Physics, 2024. Written by Yuzhou Zhuang and 10 co-authors. Institution: School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China. Major MEDLINE topics: Glioma, Magnetic Resonance Imaging, Brain Neoplasms, Imaging, Three-Dimensional. 60 references. Funded by: Wuhan Key Research and Development Program, Wuhan Major Science and Technology Project for Breaking Through the Key Bottleneck Technologies, HUST-UIH-IRI Joint Laboratory Project for Medical Imaging Technology Research and Medical Product Development.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Background

Precise glioma segmentation from multi‐parametric magnetic resonance (MR) images is essential for brain glioma diagnosis. However, due to the indistinct boundaries between tumor sub‐regions and the heterogeneous appearances of gliomas in volumetric MR scans, designing a reliable and automated glioma segmentation method is still challenging. Although existing 3D Transformer‐based or convolution‐based segmentation networks have obtained promising results via multi‐modal feature fusion strategies or contextual learning methods, they widely lack the capability of hierarchical interactions between different modalities and cannot effectively learn comprehensive feature representations related to all glioma sub‐regions.

Purpose

To overcome these problems, in this paper, we propose a 3D hierarchical cross‐modality interaction network (HCMINet) using Transformers and convolutions for accurate multi‐modal glioma segmentation, which leverages an effective hierarchical cross‐modality interaction strategy to sufficiently learn modality‐specific and modality‐shared knowledge correlated to glioma sub‐region segmentation from multi‐parametric MR images.

Methods

In the HCMINet, we first design a hierarchical cross‐modality interaction Transformer (HCMITrans) encoder to hierarchically encode and fuse heterogeneous multi‐modal features by Transformer‐based intra‐modal embeddings and inter‐modal interactions in multiple encoding stages, which effectively captures complex cross‐modality correlations while modeling global contexts. Then, we collaborate an HCMITrans encoder with a modality‐shared convolutional encoder to construct the dual‐encoder architecture in the encoding stage, which can learn the abundant contextual information from global and local perspectives. Finally, in the decoding stage, we present a progressive hybrid context fusion (PHCF) decoder to progressively fuse local and global features extracted by the dual‐encoder architecture, which utilizes the local‐global context fusion (LGCF) module to efficiently alleviate the contextual discrepancy among the decoding features.

Results

Extensive experiments are conducted on two public and competitive glioma benchmark datasets, including the BraTS2020 dataset with 494 patients and the BraTS2021 dataset with 1251 patients. Results show that our proposed method outperforms existing Transformer‐based and CNN‐based methods using other multi‐modal fusion strategies in our experiments. Specifically, the proposed HCMINet achieves state‐of‐the‐art mean DSC values of 85.33% and 91.09% on the BraTS2020 online validation dataset and the BraTS2021 local testing dataset, respectively.

Conclusions

Our proposed method can accurately and automatically segment glioma regions from multi‐parametric MR images, which is beneficial for the quantitative analysis of brain gliomas and helpful for reducing the annotation burden of neuroradiologists.

Subjects

MeSH terms (MEDLINE)

GliomaMagnetic Resonance ImagingBrain NeoplasmsImaging, Three-DimensionalHumansImage Processing, Computer-AssistedNeural Networks, Computer

Author keywords

MR imagesbrain glioma segmentationcontextual information learningmulti‐modal feature fusionvision transformer

Authors and institutions

  • School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China.
  • Wuhan Zhongke Industrial Research Institute of Medical Science Co., Ltd, Wuhan, China.
  • Wuhan United Imaging Healthcare Surgical Technology Co., Ltd, Wuhan, China.
  • School of Art and Design, Hubei University of Technology, Wuhan, China.

Funding

  • Wuhan Key Research and Development Program
  • Wuhan Major Science and Technology Project for Breaking Through the Key Bottleneck Technologies
  • HUST-UIH-IRI Joint Laboratory Project for Medical Imaging Technology Research and Medical Product Development

Cite

APA

Zhuang, Y., Liu, H., Fang, W., Ma, G., Sun, S., Zhu, Y., Zhang, X., Ge, C., Chen, W., Long, J., & Song, E. (2024). A 3D hierarchical cross‐modality interaction network using transformers and convolutions for brain glioma segmentation in MR images. Medical Physics. https://doi.org/10.1002/mp.17354

Vancouver

Zhuang Y, Liu H, Fang W, Ma G, Sun S, Zhu Y, et al. A 3D hierarchical cross‐modality interaction network using transformers and convolutions for brain glioma segmentation in MR images. Medical Physics. 2024. doi:10.1002/mp.17354

BibTeX
@article{Zhuang2024,
  author = {Zhuang, Yuzhou and Liu, Hong and Fang, Wei and Ma, Guangzhi and Sun, Sisi and Zhu, Yunfeng and Zhang, Xu and Ge, Chuanbin and Chen, Wenyang and Long, Jiaosong and Song, Enmin},
  title = {A 3D hierarchical cross‐modality interaction network using transformers and convolutions for brain glioma segmentation in MR images},
  journal = {Medical Physics},
  year = {2024},
  doi = {10.1002/mp.17354}
}
Start for free →