EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Multimodal deep learning for bone tumor diagnosis with clinical imaging, pathology, and blood biomarkers

Hang Sang, Tao Lin, Lincong Luo, Mingrui Liu, Jiaying Li, Xiang Luo, Jianlin Shen, Shizhen Zhong, Lin Xu, Wenhua Huang

Journal of Bone Oncology · 2025-12 · 55 · 100718

DOI: 10.1016/j.jbo.2025.100718 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Bone Oncology, 2025-12. Verfasst von Hang Sang und 9 weiteren Autor:innen. Beteiligte Einrichtung: Guangdong Engineering Research Center for Translation of Medical 3D Printing Application, Guangdong Provincial Key Laboratory of Digital Medicine and…. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: China Postdoctoral Science Foundation, Laboratory on Development and Application of Rural Renewable Energy Ministry of Agriculture.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurate classification of bone tumors as benign, malignant, or intermediate is crucial for patient treatment decisions. Misclassification may result in overtreatment of benign cases or delayed intervention for aggressive tumors, significantly impacting patient prognosis. However, current methods rely heavily on single-modality imaging analysis, making it difficult to handle variable lesion locations and complex cancer types. To address these limitations, we propose a novel multimodal deep learning framework that integrates clinical images, pathological slices, and blood biomarkers for automated bone tumor detection and three-class classification. The framework operates in two stages: first, a YOLOv5-based detection model localizes tumor regions on clinical images. Next, a classification model utilizes ResNet to extract deep features from both the clinical images and pathological slices, while abnormal blood biomarkers are transformed into descriptive text by a large language model and subsequently encoded into semantic features using BioBERT. Finally, features from all three modalities are integrated via a fusion module to capture complementary information and enable accurate tumor classification. The evaluation was performed using two distinct datasets: a clinical imaging dataset for bone tumor detection, and a separate multi-modal cohort comprising clinical imaging, pathology, and blood biomarkers for tumor classification. The detection model demonstrated strong localization capabilities, achieving a test mAP@0.5 of 0.7925. For the classification task, ablation studies validated the complementary contribution of each modality. Notably, our multimodal fusion approach outperformed unimodal baselines, attaining a macro-average precision of 0.9056, F1-score of 0.8736, and AUC of 0.9759 in tumor classification-outperforming existing models.

Schlagwörter

Stichwörter der Autor:innen

Bone tumorClinical decision supportDetection and classificationMultimodal deep learning

Autor:innen und Institutionen

  • Guangdong Engineering Research Center for Translation of Medical 3D Printing Application, Guangdong Provincial Key Laboratory of Digital Medicine and Biomechanics, National Key Discipline of Human Anatomy, School of Bas…
  • Yue Bei People's Hospital Postdoctoral Innovation Practice Base, Southern Medical University, Guangzhou, 510515, Guangdong, China.
  • School of Basic Medicine, Dali University, Dali, 671003, Yunnan, China.
  • Guangxi Clinical Research Center for Digital Medicine and 3D Printing, Guigang City People's Hospital, Guigang, 537100, Guangxi, China.
  • Department of Orthopaedics, Affiliated Hospital of Putian University, Putian, 351100, Fujian, China.
  • School of Basic Medical Sciences, Southern Medical University, Guangzhou, 510515, Guangdong, China.

Förderung

  • China Postdoctoral Science Foundation
  • Laboratory on Development and Application of Rural Renewable Energy Ministry of Agriculture

Zitieren

APA

Sang, H., Lin, T., Luo, L., Liu, M., Li, J., Luo, X., Shen, J., Zhong, S., Xu, L., & Huang, W. (2025). Multimodal deep learning for bone tumor diagnosis with clinical imaging, pathology, and blood biomarkers. Journal of Bone Oncology, 55, 100718. https://doi.org/10.1016/j.jbo.2025.100718

Vancouver

Sang H, Lin T, Luo L, Liu M, Li J, Luo X, et al. Multimodal deep learning for bone tumor diagnosis with clinical imaging, pathology, and blood biomarkers. Journal of Bone Oncology. 2025;55:100718. doi:10.1016/j.jbo.2025.100718

BibTeX
@article{Sang2025,
  author = {Sang, Hang and Lin, Tao and Luo, Lincong and Liu, Mingrui and Li, Jiaying and Luo, Xiang and Shen, Jianlin and Zhong, Shizhen and Xu, Lin and Huang, Wenhua},
  title = {Multimodal deep learning for bone tumor diagnosis with clinical imaging, pathology, and blood biomarkers},
  journal = {Journal of Bone Oncology},
  year = {2025},
  volume = {55},
  pages = {100718},
  doi = {10.1016/j.jbo.2025.100718}
}
Kostenlos starten →