EUVIMEDEUVIMED entdecken
Crossref

Deep learning multimodal fusion model based on ultrasound images and clinical-ultrasound texts for differentiating benign and malignant renal tumors

Hongli Cao, Yilei Shi, Shumeng Yu, Xu Cao, Chen Li, Lichao Mou, Jingliang Hu, Lixia Yan, Pei Sun, Cuixian Li, Qing Lu, Jiexian Wen und 6 weitere

British Journal of Radiology · 22.10.2026

DOI: 10.1093/bjr/tqag238 ↗

Kurz gefasst

Fachartikel in der Zeitschrift British Journal of Radiology, online erschienen am 22.10.2026. Verfasst von Hongli Cao und 17 weiteren Autor:innen. Beteiligte Einrichtung: Shanghai Institute of Infectious Disease and Biosecurity, Fudan University, Shanghai, China. Die Arbeit nennt 32 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

To develop a deep learning multimodal fusion model based on ultrasound (US) images and clinical-US texts to differentiate benign and malignant renal tumors.

Methods

The retrospective dataset from January 2012 to February 2023 in a tertiary hospital was used for model development and randomly allocated to training and validation sets. An independent prospective test set was obtained from March 2023 to October 2024. We developed Model-G using grayscale US (GUS) images, Model-GC using GUS and color Doppler flow imaging images, Model-T using clinical-US texts, and Model-UT using US images and clinical-US texts. ResNet50 was used for image feature extraction, BERT for text encoding, and ViT for multimodal classification.

Results

In total, 1008 patients (740 in training set, 186 validation set, and 82 prospective test set) with a single renal tumor were included. In the validation set, Model-UT achieved the highest diagnostic performance in comparison with Model-G, Model-GC, and Model-T with an area under the receiver operating characteristic curve (AUC) of 0.89 (95% CI, 0.86-0.93) (all P values <.05). In the prospective test set, Model-UT yielded an AUC of 0.92 (95% CI, 0.86-0.97), outperforming high-level, mid-level, and low-level radiologists (all P values <.05).

Conclusions

The multimodal fusion model presents promising potential in differentiating benign and malignant renal tumors. Advances in knowledge Deep learning model integrating US images with clinical-US texts improved benign-vs-malignant renal tumor differentiation. Decision-curve, controlled ablation, and SHAP analyses support its potential use as a clinical decision-support tool.

Schlagwörter

Stichwörter der Autor:innen

deep learningdiagnosismultimodal fusionrenal tumorsultrasound

Autor:innen und Institutionen

  • Shanghai Institute of Infectious Disease and Biosecurity, Fudan University, Shanghai, China.
  • Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China.
  • MedAI Technology (Wuxi) Co., Ltd. Wuxi, China.
  • Department of Biomedical Engineering, School of Information Science and Technology, Fudan University, Shanghai, China.
  • Shanghai Institute of Imaging Medicine, Shanghai, China.
  • State Key Laboratory of Integrated Chips and System, Fudan University, Shanghai, China.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Cao, H., Shi, Y., Yu, S., Cao, X., Li, C., Mou, L., Hu, J., Yan, L., Sun, P., Li, C., Lu, Q., Wen, J., Zhang, Q., Ta, D., Zhu, X., Xu, H., Zhao, C., & Huang, B. (2026). Deep learning multimodal fusion model based on ultrasound images and clinical-ultrasound texts for differentiating benign and malignant renal tumors. British Journal of Radiology. https://doi.org/10.1093/bjr/tqag238

Vancouver

Cao H, Shi Y, Yu S, Cao X, Li C, Mou L, et al. Deep learning multimodal fusion model based on ultrasound images and clinical-ultrasound texts for differentiating benign and malignant renal tumors. British Journal of Radiology. 2026. doi:10.1093/bjr/tqag238

BibTeX
@article{Cao2026,
  author = {Cao, Hongli and Shi, Yilei and Yu, Shumeng and Cao, Xu and Li, Chen and Mou, Lichao and Hu, Jingliang and Yan, Lixia and Sun, Pei and Li, Cuixian and Lu, Qing and Wen, Jiexian and Zhang, Qi and Ta, Dean and Zhu, Xiaoxiang and Xu, Huixiong and Zhao, Chongke and Huang, Beijian},
  title = {Deep learning multimodal fusion model based on ultrasound images and clinical-ultrasound texts for differentiating benign and malignant renal tumors},
  journal = {British Journal of Radiology},
  year = {2026},
  doi = {10.1093/bjr/tqag238}
}
Kostenlos starten →