Multi‐modality multiorgan image segmentation using continual learning with enhanced hard attention to the task
Medical Physics · 2025
DOI: 10.1002/mp.17842 ↗Kurz gefasst
Fachartikel in der Zeitschrift Medical Physics, 2025. Verfasst von Ming-Long Wu und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan. Hauptthemen laut MEDLINE: Image Processing, Computer-Assisted, Multimodal Imaging, Deep Learning, Machine Learning. Die Arbeit nennt 48 Literaturangaben. Gefördert von: National Science and Technology Council, Metal Industries Research and Development Centre.
Abstract (Originalsprache)
Background
Enabling a deep neural network (DNN) to learn multiple tasks using the concept of continual learning potentially better mimics human brain functions. However, current continual learning studies for medical image segmentation are mostly limited to single‐modality images at identical anatomical locations.
Purpose
To propose and evaluate a continual learning method termed eHAT (enhanced hard attention to the task) for performing multi‐modality, multiorgan segmentation tasks using a DNN.
Methods
Four public datasets covering the lumbar spine, heart, and brain acquired by magnetic resonance imaging (MRI) and computed tomography (CT) were included to segment the vertebral bodies, the right ventricle, and brain tumors, respectively. Three‐task (spine CT, heart MRI, and brain MRI) and four‐task (spine CT, heart MRI, brain MRI, and spine MRI) models were tested for eHAT, with the three‐task results compared with state‐of‐the‐art continual learning methods. The effectiveness of multitask performance was measured using the forgetting rate, defined as the average difference in Dice coefficients and Hausdorff distances between multiple‐task and single‐task models. The ability to transfer knowledge to different tasks was evaluated using backward transfer (BWT).
Results
The forgetting rates were −2.51% to −0.60% for the three‐task eHAT models with varying task orders, substantially better than the −18.13% to −3.59% using original hard attention to the task (HAT), while those in four‐task models were −2.54% to −1.59%. In addition, four‐task U‐net models with eHAT using only half the number of channels (1/4 parameters) yielded nearly equal performance with or without regularization. A retrospective model comparison showed that eHAT with fixed or automatic regularization had significantly superior BWT (−3% to 0%) compared to HAT (−22% to −4%).
Conclusion
We demonstrate for the first time that eHAT effectively achieves continual learning of multi‐modality, multiorgan segmentation tasks using a single DNN, with improved forgetting rates compared with HAT.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan.
- Institute of Medical Informatics, National Cheng Kung University, Tainan, Taiwan.
Förderung
- National Science and Technology Council
- Metal Industries Research and Development Centre
Zitieren
APA
Wu, M. L., & Peng, Y. F. (2025). Multi‐modality multiorgan image segmentation using continual learning with enhanced hard attention to the task. Medical Physics. https://doi.org/10.1002/mp.17842
Vancouver
Wu ML, Peng YF. Multi‐modality multiorgan image segmentation using continual learning with enhanced hard attention to the task. Medical Physics. 2025. doi:10.1002/mp.17842
BibTeX
@article{Wu2025,
author = {Wu, Ming-Long and Peng, Yi-Fan},
title = {Multi‐modality multiorgan image segmentation using continual learning with enhanced hard attention to the task},
journal = {Medical Physics},
year = {2025},
doi = {10.1002/mp.17842}
}
EUVIMED