EUVIMEDObjevte EUVIMED
Crossref

Multi‐modality multiorgan image segmentation using continual learning with enhanced hard attention to the task

Ming-Long Wu, Yi-Fan Peng

Medical Physics · 2025

DOI: 10.1002/mp.17842 ↗

At a glance

Journal article in Medical Physics, 2025. Written by Ming-Long Wu and 1 co-authors. Institution: Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan. Major MEDLINE topics: Image Processing, Computer-Assisted, Multimodal Imaging, Deep Learning, Machine Learning. 48 references. Funded by: National Science and Technology Council, Metal Industries Research and Development Centre.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Enabling a deep neural network (DNN) to learn multiple tasks using the concept of continual learning potentially better mimics human brain functions. However, current continual learning studies for medical image segmentation are mostly limited to single‐modality images at identical anatomical locations.

Purpose

To propose and evaluate a continual learning method termed eHAT (enhanced hard attention to the task) for performing multi‐modality, multiorgan segmentation tasks using a DNN.

Methods

Four public datasets covering the lumbar spine, heart, and brain acquired by magnetic resonance imaging (MRI) and computed tomography (CT) were included to segment the vertebral bodies, the right ventricle, and brain tumors, respectively. Three‐task (spine CT, heart MRI, and brain MRI) and four‐task (spine CT, heart MRI, brain MRI, and spine MRI) models were tested for eHAT, with the three‐task results compared with state‐of‐the‐art continual learning methods. The effectiveness of multitask performance was measured using the forgetting rate, defined as the average difference in Dice coefficients and Hausdorff distances between multiple‐task and single‐task models. The ability to transfer knowledge to different tasks was evaluated using backward transfer (BWT).

Results

The forgetting rates were −2.51% to −0.60% for the three‐task eHAT models with varying task orders, substantially better than the −18.13% to −3.59% using original hard attention to the task (HAT), while those in four‐task models were −2.54% to −1.59%. In addition, four‐task U‐net models with eHAT using only half the number of channels (1/4 parameters) yielded nearly equal performance with or without regularization. A retrospective model comparison showed that eHAT with fixed or automatic regularization had significantly superior BWT (−3% to 0%) compared to HAT (−22% to −4%).

Conclusion

We demonstrate for the first time that eHAT effectively achieves continual learning of multi‐modality, multiorgan segmentation tasks using a single DNN, with improved forgetting rates compared with HAT.

Subjects

MeSH terms (MEDLINE)

Image Processing, Computer-AssistedMultimodal ImagingDeep LearningMachine LearningHumansMagnetic Resonance ImagingBrainHeartNeural Networks, ComputerTomography, X-Ray Computed

Author keywords

continual learninghard attention to the taskimage segmentationmedical image

Authors and institutions

  • Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan.
  • Institute of Medical Informatics, National Cheng Kung University, Tainan, Taiwan.

Funding

  • National Science and Technology Council
  • Metal Industries Research and Development Centre

Cite

APA

Wu, M. L., & Peng, Y. F. (2025). Multi‐modality multiorgan image segmentation using continual learning with enhanced hard attention to the task. Medical Physics. https://doi.org/10.1002/mp.17842

Vancouver

Wu ML, Peng YF. Multi‐modality multiorgan image segmentation using continual learning with enhanced hard attention to the task. Medical Physics. 2025. doi:10.1002/mp.17842

BibTeX
@article{Wu2025,
  author = {Wu, Ming-Long and Peng, Yi-Fan},
  title = {Multi‐modality multiorgan image segmentation using continual learning with enhanced hard attention to the task},
  journal = {Medical Physics},
  year = {2025},
  doi = {10.1002/mp.17842}
}
Start for free →