EUVIMEDDescoperă EUVIMED
Crossref

Multiple attention channels aggregated network for multimodal medical image fusion

Jingxue Huang, Tianshu Tan, Xiaosong Li, Tao Ye, Yanxiong Wu

Medical Physics · 2025-04 · 52 (4) · 2356-2374

DOI: 10.1002/mp.17607 ↗

At a glance

Journal article in Medical Physics, published online on 2024-12-27. Written by Jingxue Huang and 4 co-authors. Institution: School of Physics and Optoelectronic Engineering, Foshan University, Foshan, China. Major MEDLINE topics: Multimodal Imaging, Image Processing, Computer-Assisted. Cited 10 times so far (Crossref); 62 references. Funded by: the Basic and Applied Basic Research of Guangdong Province, the Natural Science Foundation of Guangdong Province, the National Natural Science Foundation of China.

RezumatSubjectsDate bibliograficeCiteRelated studies

Abstract (original language)

Background

In clinical practices, doctors usually need to synthesize several single‐modality medical images for diagnosis, which is a time‐consuming and costly process. With this background, multimodal medical image fusion (MMIF) techniques have emerged to synthesize medical images of different modalities, providing a comprehensive and objective interpretation of the lesion.

Purpose

Although existing MMIF approaches have shown promising results, they often overlook the importance of multiscale feature diversity and attention interaction, which are essential for superior visual outcomes. This oversight can lead to diminished fusion performance. To bridge the gaps, we introduce a novel approach that emphasizes the integration of multiscale features through a structured decomposition and attention interaction.

Methods

Our method first decomposes the source images into three distinct groups of multiscale features by stacking different numbers of diverse branch blocks. Then, to extract global and local information separately for each group of features, we designed the convolutional and Transformer block attention branch. These two attention branches make full use of channel and spatial attention mechanisms and achieve attention interaction, enabling the corresponding feature channels to fully capture local and global information and achieve effective inter‐block feature aggregation.

Results

For the MRI‐PET fusion type, MACAN achieves average improvements of 24.48%, 27.65%, 19.24%, 27.32%, 18.51%, and 10.33% over the compared methods in terms of Q cb , AG, SSIM, SF, Q abf , and VIF metrics, respectively. Similarly, for the MRI‐SPECT fusion type, MACAN outperforms the compared methods with average improvements of 29.13%, 26.43%, 18.20%, 27.71%, 16.79%, and 10.38% in the same metrics. In addition, our method demonstrates promising results in segmentation experiments. Specifically, for the T2‐T1ce fusion, it achieves a Dice coefficient of 0.60 and a Hausdorff distance of 15.15. Comparable performance is observed for the Flair‐T1ce fusion, with a Dice coefficient of 0.60 and a Hausdorff distance of 13.27.

Conclusion

The proposed multiple attention channels aggregated network (MACAN) can effectively retain the complementary information from source images. The evaluation of MACAN through medical image fusion and segmentation experiments on public datasets demonstrated its superiority over the state‐of‐the‐art methods, both in terms of visual quality and objective metrics. Our code is available at https://github.com/JasonWong30/MACAN.

Subjects

MeSH terms (MEDLINE)

Multimodal ImagingImage Processing, Computer-AssistedHumansMagnetic Resonance ImagingPositron-Emission Tomography

Author keywords

attention interactionmultimodal medical image fusionmultiscale features

Authors and institutions

  • School of Physics and Optoelectronic Engineering, Foshan University, Foshan, China.
  • School of Engineering, Hong Kong University of Science and Technology, Kowloon, Hong Kong, China.
  • Guangdong Provincial Key Laboratory of Industrial Intelligent Inspection Technology, Foshan University, Foshan, China.
  • Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, Foshan University, Foshan, China.
  • School of Mechanical Electronic and Information Engineering, China University of Mining and Technology, Beijing, China.

Funding

  • the Basic and Applied Basic Research of Guangdong Province
  • the Natural Science Foundation of Guangdong Province
  • the National Natural Science Foundation of China
  • the Guangdong Higher Education Innovation and Strengthening of Universities Project
  • the Research Fund of Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology

Cite

APA

Huang, J., Tan, T., Li, X., Ye, T., & Wu, Y. (2025). Multiple attention channels aggregated network for multimodal medical image fusion. Medical Physics, 52(4), 2356-2374. https://doi.org/10.1002/mp.17607

Vancouver

Huang J, Tan T, Li X, Ye T, Wu Y. Multiple attention channels aggregated network for multimodal medical image fusion. Medical Physics. 2025;52(4):2356-2374. doi:10.1002/mp.17607

BibTeX
@article{Huang2025,
  author = {Huang, Jingxue and Tan, Tianshu and Li, Xiaosong and Ye, Tao and Wu, Yanxiong},
  title = {Multiple attention channels aggregated network for multimodal medical image fusion},
  journal = {Medical Physics},
  year = {2025},
  volume = {52},
  number = {4},
  pages = {2356--2374},
  doi = {10.1002/mp.17607}
}
Start for free →