EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access

A Vision-Language–Guided Multimodal Fusion Network for Glottic Carcinoma Early Diagnosis: Model Development and Validation Study

Zhaohui Jin iD, Yi Shuai iD, Yun Li iD, Mianmian Chen iD, Yumeng Liu iD, Wenbin Lei iD, Xiaomao Fan iD

JMIR Medical Informatics · 2025

DOI: 10.2196/74902 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift JMIR Medical Informatics, 2025. Verfasst von Zhaohui Jin und 6 weiteren Autor:innen. Beteiligte Einrichtung: College of Big Data and Internet, Shenzhen Technology University, Pingshan District, 3002 Lantian Road, Shenzhen, Guangdong, 518118, China, 86 192766…. Hauptthemen laut MEDLINE: Laryngeal Neoplasms, Glottis, Early Detection of Cancer. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Early diagnosis and intervention in glottic carcinoma (GC) can significantly improve long-term prognosis. However, the accurate diagnosis of early GC is challenging due to its morphological similarity to vocal cord dysplasia, with the difficulty further exacerbated in medically underserved areas.

Objective

This study aims to address the limitations of existing technologies by designing a vision-language multimodal model, providing a more efficient and accurate early diagnostic method for GC.

Methods

The data used in this study were sourced from the information system of the First Affiliated Hospital of Sun Yat-sen University, comprising laryngoscopy reports and 5796 laryngoscopic images from 404 patients with glottic lesions. We propose a vision-language–guided multimodal fusion network (VLMF-Net) based on a large vision-language model for the early automated diagnosis of GC. The text processing module of this model uses the pretrained Large Language Model Meta AI (LLaMa) to generate text vector representations, while the image processing module uses a pretrained vision transformer to extract features from laryngoscopic images, achieving cross-modal alignment through the Q-Former module. By leveraging a feature fusion module, deep integration of text and image features is achieved, ultimately enabling classification diagnosis. To validate the model’s performance, the study selected contrastive language-image pretraining (CLIP), bootstrapping language-image pretraining with frozen image encoders and large language models (BLIP-2), a large-scale image and noisy-text embedding (ALIGN), and vision-and-language transformer (VILT) as baseline methods for experimental evaluation on the same dataset, with comprehensive performance assessment conducted using accuracy, recall, precision, F 1 -score, and area under the curve.

Results

We found that on the internal test set, the VLMF-Net model significantly outperformed existing methods with an accuracy of 77.6% (CLIP: 70.5%; BLIP-2: 71.5%; ALIGN: 67.3%; and VILT: 64.3%), achieving a 6.1-percentage point improvement over the best baseline model (BLIP-2). On the external test set, our method also demonstrated robust performance, achieving an accuracy of 73.9%, which is 4.6 percentage points higher than the second-best model (BLIP-2: 69.3%). This indicates that our model surpasses these methods in the early diagnosis of GC and exhibits strong generalization ability and robustness.

Conclusions

The proposed VLMF-Net model can be effectively used for the early diagnosis of GC, helping to address the challenges in its early detection.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Laryngeal NeoplasmsGlottisEarly Detection of CancerHumansNeural Networks, ComputerLaryngoscopy

Stichwörter der Autor:innen

clinical decision makingcomputer-aided diagnosisglottic carcinoma early diagnosislarge-scale foundation modelmultimodal deep learning

Autor:innen und Institutionen

  • College of Big Data and Internet, Shenzhen Technology University, Pingshan District, 3002 Lantian Road, Shenzhen, Guangdong, 518118, China, 86 19276679344.
  • The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China.

ORCID-Profile: Zhaohui Jin, Yi Shuai, Yun Li, Mianmian Chen, Yumeng Liu, Wenbin Lei, Xiaomao Fan

Zitieren

APA

Jin, Z., Shuai, Y., Li, Y., Chen, M., Liu, Y., Lei, W., & Fan, X. (2025). A Vision-Language–Guided Multimodal Fusion Network for Glottic Carcinoma Early Diagnosis: Model Development and Validation Study. JMIR Medical Informatics. https://doi.org/10.2196/74902

Vancouver

Jin Z, Shuai Y, Li Y, Chen M, Liu Y, Lei W, et al. A Vision-Language–Guided Multimodal Fusion Network for Glottic Carcinoma Early Diagnosis: Model Development and Validation Study. JMIR Medical Informatics. 2025. doi:10.2196/74902

BibTeX
@article{Jin2025,
  author = {Jin, Zhaohui and Shuai, Yi and Li, Yun and Chen, Mianmian and Liu, Yumeng and Lei, Wenbin and Fan, Xiaomao},
  title = {A Vision-Language–Guided Multimodal Fusion Network for Glottic Carcinoma Early Diagnosis: Model Development and Validation Study},
  journal = {JMIR Medical Informatics},
  year = {2025},
  doi = {10.2196/74902}
}
Kostenlos starten →