Bridging the Semantic Gap in Medical Visual Question Answering With Prompt Learning
IEEE Transactions on Medical Imaging · 2025
DOI: 10.1109/tmi.2025.3580561 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2025. Verfasst von Zilin Lu und 4 weiteren Autor:innen. Hauptthemen laut MEDLINE: Semantics, Machine Learning, Diagnostic Imaging, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Ningbo Clinical Research Center for Medical Imaging, Shenzhen Science and Technology Program.
Abstract (Originalsprache)
Medical Visual Question Answering (Med-VQA) aims to answer questions regarding the content of medical images, crucial for enhancing diagnostics and education in healthcare. However, progress in this field is hindered by data scarcity due to the resource-intensive nature of medical data annotation. While existing Med-VQA approaches often rely on pre-training to mitigate this issue, bridging the semantic gap between pre-trained models and specific tasks remains a significant challenge. This paper presents the Dynamic Semantic-Adaptive Prompting (DSAP) framework, leveraging prompt learning to enhance model performance in Med-VQA. To this end, we introduce two prompting strategies: Semantic Alignment Prompting (SAP) and Dynamic Question-Aware Prompting (DQAP). SAP prompts multi-modal inputs during fine-tuning, reducing the semantic gap by aligning model outputs with domain-specific contexts. Simultaneously, DQAP enhances answer selection by leveraging grammatical relationships between questions and answers, thereby improving accuracy and relevance. The DSAP framework was pre-trained on three datasets-ROCO, MedICaT, and MIMIC-CXR-and comprehensively evaluated against 15 existing Med-VQA models on three public datasets: VQA-RAD, SLAKE, and PathVQA. Our results demonstrate a substantial performance improvement, with DSAP achieving a 1.9% enhancement in average results across benchmarks. These findings underscore DSAP's effectiveness in addressing critical challenges in Med-VQA and suggest promising avenues for future developments in medical AI.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- National Natural Science Foundation of China
- Ningbo Clinical Research Center for Medical Imaging
- Shenzhen Science and Technology Program
Zitieren
APA
Lu, Z., Zeng, Q., Lu, M., Chen, G., & Xia, Y. (2025). Bridging the Semantic Gap in Medical Visual Question Answering With Prompt Learning. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/tmi.2025.3580561
Vancouver
Lu Z, Zeng Q, Lu M, Chen G, Xia Y. Bridging the Semantic Gap in Medical Visual Question Answering With Prompt Learning. IEEE Transactions on Medical Imaging. 2025. doi:10.1109/tmi.2025.3580561
BibTeX
@article{Lu2025,
author = {Lu, Zilin and Zeng, Qingjie and Lu, Mengkang and Chen, Geng and Xia, Yong},
title = {Bridging the Semantic Gap in Medical Visual Question Answering With Prompt Learning},
journal = {IEEE Transactions on Medical Imaging},
year = {2025},
doi = {10.1109/tmi.2025.3580561}
}
EUVIMED