EUVIMEDEUVIMED entdecken
Crossref

Multi-Modal Vision–Language Transformer for Integrating Nano-Resolution Medical Imaging and Clinical Text in Virtual Diagnostic Assistants

Mariam Fawzy Kamel, Neda Rahimi

Journal of Nano Molecular Intelligence and Virtual Health Systems · 30.06.2025 · 1 (2)

DOI: 10.70023/nmivhs.122 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Nano Molecular Intelligence and Virtual Health Systems, online erschienen am 30.06.2025. Verfasst von Mariam Fawzy Kamel und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Nano-resolution medical imaging and clinical text integration still face significant challenges in intelligent healthcare due to the heterogeneity of data representations, the lack of cross-modal interaction, and inadequate interpretability in diagnosis. Most diagnostic systems available today manage visual and textual data separately, resulting in limited clinical understanding and reduced diagnostic certainty. In this paper, a new Hierarchical Cross-Attention Vision–Language Transformer with Graph Knowledge Fusion (HCVLT-GKF) is proposed to address these drawbacks in multimodal disease diagnosis in virtual diagnostic assistants. The proposed framework integrates two approaches: a Vision–Language Transformer (VLT) that uses hierarchical cross-attention to learn unified representations from nano-resolution medical images and clinical narratives, and a Graph Knowledge Fusion (GKF) module that incorporates structured medical knowledge to model relationships among symptoms, biomarkers, and diseases. The VLT contributes to the semantic congruence between the visual and textual modalities, whereas the GKF module can help improve the ability to make diagnostic deductions and enable contextual understanding. Experimental validation shows that the proposed HCVLT-GKF achieves 99.12% accuracy, 98.94% precision, 98.85% recall, 98.89% F1-score, and 99.28% AUC, which are better than those of existing multimodal diagnostic models. Moreover, the framework offers better interpretability through knowledge-guided reasoning. The findings show that HCVLT-GKF provides an effective and understandable approach to multimodal clinical decision-making, supporting and enabling the design of next-generation virtual diagnostic assistants for precision healthcare.

Zitieren

APA

Kamel, M. F., & Rahimi, N. (2025). Multi-Modal Vision–Language Transformer for Integrating Nano-Resolution Medical Imaging and Clinical Text in Virtual Diagnostic Assistants. Journal of Nano Molecular Intelligence and Virtual Health Systems, 1(2). https://doi.org/10.70023/nmivhs.122

Vancouver

Kamel MF, Rahimi N. Multi-Modal Vision–Language Transformer for Integrating Nano-Resolution Medical Imaging and Clinical Text in Virtual Diagnostic Assistants. Journal of Nano Molecular Intelligence and Virtual Health Systems. 2025;1(2). doi:10.70023/nmivhs.122

BibTeX
@article{Kamel2025,
  author = {Kamel, Mariam Fawzy and Rahimi, Neda},
  title = {Multi-Modal Vision–Language Transformer for Integrating Nano-Resolution Medical Imaging and Clinical Text in Virtual Diagnostic Assistants},
  journal = {Journal of Nano Molecular Intelligence and Virtual Health Systems},
  year = {2025},
  volume = {1},
  number = {2},
  doi = {10.70023/nmivhs.122}
}
Kostenlos starten →