EUVIMEDEUVIMED entdecken
Crossref

Medical semantic knowledge‐integrated multitask learning network for report generation and neoadjuvant chemotherapy response prediction

Wei Song, Ming Fan, Lihua Li

Medical Physics · 2025

DOI: 10.1002/mp.17925 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Medical Physics, 2025. Verfasst von Wei Song und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China. Hauptthemen laut MEDLINE: Semantics, Neoadjuvant Therapy, Machine Learning, Neural Networks, Computer. Die Arbeit nennt 50 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Zhejiang Provincial Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Automatically generated reports and predictions of neoadjuvant chemotherapy (NAC) responses can support physicians in clinical decision‐making. However, previous research has not thoroughly considered the connection between the detailed descriptions within these reports and their impact on predicting NAC responses.

Objectives

This study aims to develop an effective medical semantic knowledge‐integrated multitask learning (MSKMTL) network to enhance the accuracy of NAC response predictions and improve the quality of automatically generated medical reports.

Methods

We propose a MSKMTL network designed to increase report generation and NAC prediction by leveraging semantic knowledge within medical reports. The knowledge base was trained with DCE‐MRI and the corresponding reports to capture the interrelationships of paired visual‐text representations. During the training stage, semantic knowledge is encoded via a pretrained BERT model and stored in the knowledge base. We use contrastive learning to align the feature space of the image with the related report. In the inference stage, MSKMTL retrieves the stored semantic knowledge by querying image features to facilitate the generation of an impression section of the report and predict the NAC response. We evaluated the performance of the proposed network using metrics associated with both report generation and prediction tasks on a breast DCE‐MRI dataset and a public IU x‐ray dataset.

Results

For report generation, the BLEU‐1, BLEU‐4, ROUGE, and METEOR scores are 0.408, 0.186, 0.426, and 0.402, respectively. For NAC prediction, the area under the curve, recall, precision, and F1 score are 0.849, 0.750, 0.795, and 0.794, respectively. The network also showed strong performance with the IU x‐ray dataset.

Conclusion

Our study shows that, compared to other methodologies, integrating medical semantic knowledge increases prediction accuracy and generates superior reports.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SemanticsNeoadjuvant TherapyMachine LearningNeural Networks, ComputerHumansMagnetic Resonance ImagingBreast NeoplasmsKnowledge Bases

Stichwörter der Autor:innen

multi‐task learningneoadjuvant chemotherapy predictionreport generationsemantic knowledge base

Autor:innen und Institutionen

  • School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China.
  • Institute of Intelligent Biomedicine, Hangzhou Dianzi University, Hangzhou, China.

Förderung

  • National Natural Science Foundation of China
  • Zhejiang Provincial Natural Science Foundation of China

Zitieren

APA

Song, W., Fan, M., & Li, L. (2025). Medical semantic knowledge‐integrated multitask learning network for report generation and neoadjuvant chemotherapy response prediction. Medical Physics. https://doi.org/10.1002/mp.17925

Vancouver

Song W, Fan M, Li L. Medical semantic knowledge‐integrated multitask learning network for report generation and neoadjuvant chemotherapy response prediction. Medical Physics. 2025. doi:10.1002/mp.17925

BibTeX
@article{Song2025,
  author = {Song, Wei and Fan, Ming and Li, Lihua},
  title = {Medical semantic knowledge‐integrated multitask learning network for report generation and neoadjuvant chemotherapy response prediction},
  journal = {Medical Physics},
  year = {2025},
  doi = {10.1002/mp.17925}
}
Kostenlos starten →