ECSA: Mitigating Catastrophic Forgetting and Few-Shot Generalization in Medical Visual Question Answering
Tomography · 2025
DOI: 10.3390/tomography11100115 ↗Kurz gefasst
Fachartikel in der Zeitschrift Tomography, 2025. Verfasst von Qinhao Jia und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China. Hauptthemen laut MEDLINE: Natural Language Processing. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Objective
Medical Visual Question Answering (Med-VQA), a key technology that integrates computer vision and natural language processing to assist in clinical diagnosis, possesses significant potential for enhancing diagnostic efficiency and accuracy. However, its development is constrained by two major bottlenecks: weak few-shot generalization capability stemming from the scarcity of high-quality annotated data and the problem of catastrophic forgetting when continually learning new knowledge. Existing research has largely addressed these two challenges in isolation, lacking a unified framework.
Methods
To bridge this gap, this paper proposes a novel Evolvable Clinical-Semantic Alignment (ECSA) framework, designed to synergistically solve these two challenges within a single architecture. ECSA is built upon powerful pre-trained vision (BiomedCLIP) and language (Flan-T5) models, with two innovative modules at its core. First, we design a Clinical-Semantic Disambiguation Module (CSDM), which employs a novel debiased hard negative mining strategy for contrastive learning. This enables the precise discrimination of “hard negatives” that are visually similar but clinically distinct, thereby significantly enhancing the model’s representation ability in few-shot and long-tail scenarios. Second, we introduce a Prompt-based Knowledge Consolidation Module (PKC), which acts as a rehearsal-free non-parametric knowledge store. It consolidates historical knowledge by dynamically accumulating and retrieving task-specific “soft prompts,” thus effectively circumventing catastrophic forgetting without relying on past data.
Results
Extensive experimental results on four public benchmark datasets, VQA-RAD, SLAKE, PathVQA, and VQA-Med-2019, demonstrate ECSA’s state-of-the-art or highly competitive performance. Specifically, ECSA achieves excellent overall accuracies of 80.15% on VQA-RAD and 85.10% on SLAKE, while also showing strong generalization with 64.57% on PathVQA and 82.23% on VQA-Med-2019. More critically, in continual learning scenarios, the framework achieves a low forgetting rate of just 13.50%, showcasing its significant advantages in knowledge retention.
Conclusions
These findings validate the framework’s substantial potential for building robust and evolvable clinical decision support systems.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Jia, Q., Liu, S., Chen, M., Li, T., & Yang, J. (2025). ECSA: Mitigating Catastrophic Forgetting and Few-Shot Generalization in Medical Visual Question Answering. Tomography. https://doi.org/10.3390/tomography11100115
Vancouver
Jia Q, Liu S, Chen M, Li T, Yang J. ECSA: Mitigating Catastrophic Forgetting and Few-Shot Generalization in Medical Visual Question Answering. Tomography. 2025. doi:10.3390/tomography11100115
BibTeX
@article{Jia2025,
author = {Jia, Qinhao and Liu, Shuxian and Chen, Mingliang and Li, Tianyi and Yang, Jing},
title = {ECSA: Mitigating Catastrophic Forgetting and Few-Shot Generalization in Medical Visual Question Answering},
journal = {Tomography},
year = {2025},
doi = {10.3390/tomography11100115}
}
EUVIMED