Medical semantic knowledge‐integrated multitask learning network for report generation and neoadjuvant chemotherapy response prediction
Medical Physics · 2025
DOI: 10.1002/mp.17925 ↗Kurz gefasst
Fachartikel in der Zeitschrift Medical Physics, 2025. Verfasst von Wei Song und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China. Hauptthemen laut MEDLINE: Semantics, Neoadjuvant Therapy, Machine Learning, Neural Networks, Computer. Die Arbeit nennt 50 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Zhejiang Provincial Natural Science Foundation of China.
Themen: Onkologische Pflege
Abstract (Originalsprache)
Background
Automatically generated reports and predictions of neoadjuvant chemotherapy (NAC) responses can support physicians in clinical decision‐making. However, previous research has not thoroughly considered the connection between the detailed descriptions within these reports and their impact on predicting NAC responses.
Objectives
This study aims to develop an effective medical semantic knowledge‐integrated multitask learning (MSKMTL) network to enhance the accuracy of NAC response predictions and improve the quality of automatically generated medical reports.
Methods
We propose a MSKMTL network designed to increase report generation and NAC prediction by leveraging semantic knowledge within medical reports. The knowledge base was trained with DCE‐MRI and the corresponding reports to capture the interrelationships of paired visual‐text representations. During the training stage, semantic knowledge is encoded via a pretrained BERT model and stored in the knowledge base. We use contrastive learning to align the feature space of the image with the related report. In the inference stage, MSKMTL retrieves the stored semantic knowledge by querying image features to facilitate the generation of an impression section of the report and predict the NAC response. We evaluated the performance of the proposed network using metrics associated with both report generation and prediction tasks on a breast DCE‐MRI dataset and a public IU x‐ray dataset.
Results
For report generation, the BLEU‐1, BLEU‐4, ROUGE, and METEOR scores are 0.408, 0.186, 0.426, and 0.402, respectively. For NAC prediction, the area under the curve, recall, precision, and F1 score are 0.849, 0.750, 0.795, and 0.794, respectively. The network also showed strong performance with the IU x‐ray dataset.
Conclusion
Our study shows that, compared to other methodologies, integrating medical semantic knowledge increases prediction accuracy and generates superior reports.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China.
- Institute of Intelligent Biomedicine, Hangzhou Dianzi University, Hangzhou, China.
Förderung
- National Natural Science Foundation of China
- Zhejiang Provincial Natural Science Foundation of China
Zitieren
APA
Song, W., Fan, M., & Li, L. (2025). Medical semantic knowledge‐integrated multitask learning network for report generation and neoadjuvant chemotherapy response prediction. Medical Physics. https://doi.org/10.1002/mp.17925
Vancouver
Song W, Fan M, Li L. Medical semantic knowledge‐integrated multitask learning network for report generation and neoadjuvant chemotherapy response prediction. Medical Physics. 2025. doi:10.1002/mp.17925
BibTeX
@article{Song2025,
author = {Song, Wei and Fan, Ming and Li, Lihua},
title = {Medical semantic knowledge‐integrated multitask learning network for report generation and neoadjuvant chemotherapy response prediction},
journal = {Medical Physics},
year = {2025},
doi = {10.1002/mp.17925}
}
EUVIMED