EUVIMEDEUVIMED entdecken
Open Access

TAC-UNet: transformer-assisted convolutional neural network for medical image segmentation

Jingliu He, Yuqi Ma, Mingyue Yang, Wensong Yang, Chunming Wu, Shanxiong Chen iD

Quantitative Imaging in Medicine and Surgery · 2024-12 · 14 (12) · 8824-8839

DOI: 10.21037/qims-24-1229 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Quantitative Imaging in Medicine and Surgery, 2024-12. Verfasst von Jingliu He und 5 weiteren Autor:innen. Beteiligte Einrichtung: The College of Computer and Information Science, Southwest University, Chongqing, China. Die Arbeit wurde bisher 13-mal zitiert (Crossref). Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Medical image segmentation is crucial for improving healthcare outcomes. Convolutional neural networks (CNNs) have been widely applied in medical image analysis; however, their inherent inductive biases limit their ability to capture global contextual information. Vision transformer (ViT) architectures address this limitation by leveraging attention mechanisms to model global relationships; however, they typically require large-scale datasets for effective training, which is challenging in the field of medical imaging due to limited data availability. This study aimed to integrate the advantages of CNN and ViT architectures to improve segmentation performance on small-scale medical image datasets.

Methods

In this study, we established a U-shaped network architecture based on a Transformer-assisted convolutional neural network (TAC-UNet). The TAC-UNet is primarily composed of a hybrid structure integrating CNN and Transformer components. Specifically, the hybrid architecture follows a dual-path design in which the Transformer branch continuously conveys global contextual information to the CNN backbone. This allows the CNN backbone to enhance its global perception while building on the local features it extracts, thereby improving its ability to comprehend complex image structures. A channel cross-attention (CCA) module is also incorporated as a bridge between the encoder and decoder to better reconcile the semantic discrepancies between them.

Results

Detailed experiments on three public datasets were conducted. Specifically, our model was trained on 30 images from the Multi-organ Nucleus Segmentation (MoNuSeg) training dataset, 85 images from the Gland Segmentation (GlaS) training dataset, and 551 images from the Computer Vision Center Colorectal Cancer-Clinic Database (CVC-ClinicDB) dataset. We evaluated the performance of our model on the corresponding test sets. Our TAC-UNet achieved the best Dice scores (80.36%, 90.70%, and 91.81% on the MoNuSeg, GlaS, and CVC-ClinicDB datasets, respectively) of all the models. Compared to other CNN-based, Transformer-based, and hybrid methods, the TAC-UNet demonstrated significantly superior segmentation performance.

Conclusions

Our TAC-UNet model showed advanced segmentation performance on small-scale medical image datasets. The detailed experimental results showed the effectiveness of the method. Our model's code is available at: https://github.com/hejlhello/TAC-UNet.

Schlagwörter

Stichwörter der Autor:innen

Medical image segmentationTransformerconvolutional neural network (CNN)small-scale dataset

Autor:innen und Institutionen

  • The College of Computer and Information Science, Southwest University, Chongqing, China.
  • The Department of NHC Key Laboratory of Diagnosis and Treatment on Brain Functional Diseases, the First affiliated Hospital of Chongqing Medical University, Chongqing, China.

ORCID-Profile: Shanxiong Chen

Zitieren

APA

He, J., Ma, Y., Yang, M., Yang, W., Wu, C., & Chen, S. (2024). TAC-UNet: transformer-assisted convolutional neural network for medical image segmentation. Quantitative Imaging in Medicine and Surgery, 14(12), 8824-8839. https://doi.org/10.21037/qims-24-1229

Vancouver

He J, Ma Y, Yang M, Yang W, Wu C, Chen S. TAC-UNet: transformer-assisted convolutional neural network for medical image segmentation. Quantitative Imaging in Medicine and Surgery. 2024;14(12):8824-8839. doi:10.21037/qims-24-1229

BibTeX
@article{He2024,
  author = {He, Jingliu and Ma, Yuqi and Yang, Mingyue and Yang, Wensong and Wu, Chunming and Chen, Shanxiong},
  title = {TAC-UNet: transformer-assisted convolutional neural network for medical image segmentation},
  journal = {Quantitative Imaging in Medicine and Surgery},
  year = {2024},
  volume = {14},
  number = {12},
  pages = {8824--8839},
  doi = {10.21037/qims-24-1229}
}
Kostenlos starten →