TAC-UNet: transformer-assisted convolutional neural network for medical image segmentation
Quantitative Imaging in Medicine and Surgery · 2024-12 · 14 (12) · 8824-8839
DOI: 10.21037/qims-24-1229 ↗Kurz gefasst
Fachartikel in der Zeitschrift Quantitative Imaging in Medicine and Surgery, 2024-12. Verfasst von Jingliu He und 5 weiteren Autor:innen. Beteiligte Einrichtung: The College of Computer and Information Science, Southwest University, Chongqing, China. Die Arbeit wurde bisher 13-mal zitiert (Crossref). Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Medical image segmentation is crucial for improving healthcare outcomes. Convolutional neural networks (CNNs) have been widely applied in medical image analysis; however, their inherent inductive biases limit their ability to capture global contextual information. Vision transformer (ViT) architectures address this limitation by leveraging attention mechanisms to model global relationships; however, they typically require large-scale datasets for effective training, which is challenging in the field of medical imaging due to limited data availability. This study aimed to integrate the advantages of CNN and ViT architectures to improve segmentation performance on small-scale medical image datasets.
Methods
In this study, we established a U-shaped network architecture based on a Transformer-assisted convolutional neural network (TAC-UNet). The TAC-UNet is primarily composed of a hybrid structure integrating CNN and Transformer components. Specifically, the hybrid architecture follows a dual-path design in which the Transformer branch continuously conveys global contextual information to the CNN backbone. This allows the CNN backbone to enhance its global perception while building on the local features it extracts, thereby improving its ability to comprehend complex image structures. A channel cross-attention (CCA) module is also incorporated as a bridge between the encoder and decoder to better reconcile the semantic discrepancies between them.
Results
Detailed experiments on three public datasets were conducted. Specifically, our model was trained on 30 images from the Multi-organ Nucleus Segmentation (MoNuSeg) training dataset, 85 images from the Gland Segmentation (GlaS) training dataset, and 551 images from the Computer Vision Center Colorectal Cancer-Clinic Database (CVC-ClinicDB) dataset. We evaluated the performance of our model on the corresponding test sets. Our TAC-UNet achieved the best Dice scores (80.36%, 90.70%, and 91.81% on the MoNuSeg, GlaS, and CVC-ClinicDB datasets, respectively) of all the models. Compared to other CNN-based, Transformer-based, and hybrid methods, the TAC-UNet demonstrated significantly superior segmentation performance.
Conclusions
Our TAC-UNet model showed advanced segmentation performance on small-scale medical image datasets. The detailed experimental results showed the effectiveness of the method. Our model's code is available at: https://github.com/hejlhello/TAC-UNet.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- The College of Computer and Information Science, Southwest University, Chongqing, China.
- The Department of NHC Key Laboratory of Diagnosis and Treatment on Brain Functional Diseases, the First affiliated Hospital of Chongqing Medical University, Chongqing, China.
ORCID-Profile: Shanxiong Chen
Zitieren
APA
He, J., Ma, Y., Yang, M., Yang, W., Wu, C., & Chen, S. (2024). TAC-UNet: transformer-assisted convolutional neural network for medical image segmentation. Quantitative Imaging in Medicine and Surgery, 14(12), 8824-8839. https://doi.org/10.21037/qims-24-1229
Vancouver
He J, Ma Y, Yang M, Yang W, Wu C, Chen S. TAC-UNet: transformer-assisted convolutional neural network for medical image segmentation. Quantitative Imaging in Medicine and Surgery. 2024;14(12):8824-8839. doi:10.21037/qims-24-1229
BibTeX
@article{He2024,
author = {He, Jingliu and Ma, Yuqi and Yang, Mingyue and Yang, Wensong and Wu, Chunming and Chen, Shanxiong},
title = {TAC-UNet: transformer-assisted convolutional neural network for medical image segmentation},
journal = {Quantitative Imaging in Medicine and Surgery},
year = {2024},
volume = {14},
number = {12},
pages = {8824--8839},
doi = {10.21037/qims-24-1229}
}
EUVIMED