Symptom-Aware Adaptation of CLIP for Few-Shot Medical Image Classification
IEEE Signal Processing Letters · 2026 · 33 · 1701-1705
DOI: 10.1109/lsp.2026.3685174 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Signal Processing Letters, 2026. Verfasst von Zexiang Li und 6 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Gefördert von: Natural Science Foundation of Shandong Province, China Postdoctoral Science Foundation.
Abstract (Originalsprache)
Vision–language pre-training model yields powerful general-purpose representations, yet its direct application to medical images is hindered by two key gaps: (i) low sensitivity to fine-grained pathological cues and (ii) semantic misalignment with structured clinical terminology. We propose SITSA, a framework that adapts a frozen CLIP backbone for data-scarce medical tasks without full fine-tuning. SITSA introduces three synergistic components: (1) MedVisExtractor, a depthwise-separable convolutional adapter that recalibrates patch tokens to emphasize subtle lesions; (2) SymSemLoRA, which injects low-rank updates into the text encoder conditioned on large-language-model-generated prompts that encode texture, location, and shape attributes; and (3) Prototype-Guided Alignment, a cross-modal module that binds image patches and prompt tokens via shared learnable prototypes. Extensive experiments on COVID-QU-Ex, OCTMNIST, and DERMAMNIST demonstrate that SITSA surpasses thirteen state-of-the-art baselines in both few-shot and base-to-novel generalization, achieving the highest harmonic mean accuracy while keeping the backbone frozen.
Förderung
- Natural Science Foundation of Shandong Province
- China Postdoctoral Science Foundation
Zitieren
APA
Li, Z., Wang, W., Liu, C., He, J., Zhao, Y., Zhai, Y., & Tang, R. (2026). Symptom-Aware Adaptation of CLIP for Few-Shot Medical Image Classification. IEEE Signal Processing Letters, 33, 1701-1705. https://doi.org/10.1109/lsp.2026.3685174
Vancouver
Li Z, Wang W, Liu C, He J, Zhao Y, Zhai Y, et al. Symptom-Aware Adaptation of CLIP for Few-Shot Medical Image Classification. IEEE Signal Processing Letters. 2026;33:1701-1705. doi:10.1109/lsp.2026.3685174
BibTeX
@article{Li2026,
author = {Li, Zexiang and Wang, Wenxu and Liu, Chengpu and He, Jiawen and Zhao, Youwei and Zhai, Yili and Tang, Ruichun},
title = {Symptom-Aware Adaptation of CLIP for Few-Shot Medical Image Classification},
journal = {IEEE Signal Processing Letters},
year = {2026},
volume = {33},
pages = {1701--1705},
doi = {10.1109/lsp.2026.3685174}
}
EUVIMED