EUVIMEDEUVIMED entdecken
Crossref

Bridging the Semantic Gap in Medical Visual Question Answering With Prompt Learning

Zilin Lu, Qingjie Zeng, Mengkang Lu, Geng Chen, Yong Xia

IEEE Transactions on Medical Imaging · 2025

DOI: 10.1109/tmi.2025.3580561 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2025. Verfasst von Zilin Lu und 4 weiteren Autor:innen. Hauptthemen laut MEDLINE: Semantics, Machine Learning, Diagnostic Imaging, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Ningbo Clinical Research Center for Medical Imaging, Shenzhen Science and Technology Program.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Medical Visual Question Answering (Med-VQA) aims to answer questions regarding the content of medical images, crucial for enhancing diagnostics and education in healthcare. However, progress in this field is hindered by data scarcity due to the resource-intensive nature of medical data annotation. While existing Med-VQA approaches often rely on pre-training to mitigate this issue, bridging the semantic gap between pre-trained models and specific tasks remains a significant challenge. This paper presents the Dynamic Semantic-Adaptive Prompting (DSAP) framework, leveraging prompt learning to enhance model performance in Med-VQA. To this end, we introduce two prompting strategies: Semantic Alignment Prompting (SAP) and Dynamic Question-Aware Prompting (DQAP). SAP prompts multi-modal inputs during fine-tuning, reducing the semantic gap by aligning model outputs with domain-specific contexts. Simultaneously, DQAP enhances answer selection by leveraging grammatical relationships between questions and answers, thereby improving accuracy and relevance. The DSAP framework was pre-trained on three datasets-ROCO, MedICaT, and MIMIC-CXR-and comprehensively evaluated against 15 existing Med-VQA models on three public datasets: VQA-RAD, SLAKE, and PathVQA. Our results demonstrate a substantial performance improvement, with DSAP achieving a 1.9% enhancement in average results across benchmarks. These findings underscore DSAP's effectiveness in addressing critical challenges in Med-VQA and suggest promising avenues for future developments in medical AI.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SemanticsMachine LearningDiagnostic ImagingImage Processing, Computer-AssistedHumansDatabases, Factual

Förderung

  • National Natural Science Foundation of China
  • Ningbo Clinical Research Center for Medical Imaging
  • Shenzhen Science and Technology Program

Zitieren

APA

Lu, Z., Zeng, Q., Lu, M., Chen, G., & Xia, Y. (2025). Bridging the Semantic Gap in Medical Visual Question Answering With Prompt Learning. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/tmi.2025.3580561

Vancouver

Lu Z, Zeng Q, Lu M, Chen G, Xia Y. Bridging the Semantic Gap in Medical Visual Question Answering With Prompt Learning. IEEE Transactions on Medical Imaging. 2025. doi:10.1109/tmi.2025.3580561

BibTeX
@article{Lu2025,
  author = {Lu, Zilin and Zeng, Qingjie and Lu, Mengkang and Chen, Geng and Xia, Yong},
  title = {Bridging the Semantic Gap in Medical Visual Question Answering With Prompt Learning},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2025},
  doi = {10.1109/tmi.2025.3580561}
}
Kostenlos starten →