EUVIMEDEUVIMED entdecken
Crossref

Medical Knowledge Intervention Prompt Tuning for Medical Image Classification

Ye Du, Nanxi Yu, Shujun Wang

IEEE Transactions on Medical Imaging · 2025-12 · 44 (12) · 4945-4959

DOI: 10.1109/tmi.2025.3584841 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2025-12. Verfasst von Ye Du und 2 weiteren Autor:innen. Hauptthemen laut MEDLINE: Image Interpretation, Computer-Assisted, Diagnostic Imaging, Natural Language Processing. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 89 Literaturangaben. Gefördert von: Research Grants Council (RGC) Collaborative Research Fund, Start-Up Fund of The Hong Kong Polytechnic University, Seed Fund of the Research Institute for Smart Ageing.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vision-language foundation models (VLMs) have shown great potential in feature transfer and generalization across a wide spectrum of medical-related downstream tasks. However, fine-tuning these models is resource-intensive due to their large number of parameters. Prompt tuning has emerged as a viable solution to mitigate memory usage and reduce training time while maintaining competitive performance. Nevertheless, the challenge is that existing prompt tuning methods cannot precisely distinguish different kinds of medical concepts, which miss essentially specific disease-related features across various medical imaging modalities in medical image classification tasks. We find that Large Language Models (LLMs), trained on extensive text corpora, are particularly adept at providing this specialized medical knowledge. Motivated by this, we propose incorporating LLMs into the prompt tuning process. Specifically, we introduce the CILMP, Conditional Intervention of Large Language Models for Prompt Tuning, a method that bridges LLMs and VLMs to facilitate the transfer of medical knowledge into VLM prompts. CILMP extracts disease-specific representations from LLMs, intervenes within a low-rank linear subspace and utilizes them to create disease-specific prompts. Additionally, a conditional mechanism is incorporated to condition the intervention process on each individual medical image, generating instance-adaptive prompts and thus enhancing adaptability. Extensive experiments across diverse medical image datasets demonstrate that CILMP consistently outperforms state-of-the-art prompt tuning methods, demonstrating its effectiveness. Code is available at https://usr922.github.io/cilmp.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Image Interpretation, Computer-AssistedDiagnostic ImagingNatural Language ProcessingHumansAlgorithmsDatabases, Factual

Förderung

  • Research Grants Council (RGC) Collaborative Research Fund
  • Start-Up Fund of The Hong Kong Polytechnic University
  • Seed Fund of the Research Institute for Smart Ageing
  • Tsinghua-PolyU Joint Research Initiative Fund
  • PolyU UGC Funding

Zitieren

APA

Du, Y., Yu, N., & Wang, S. (2025). Medical Knowledge Intervention Prompt Tuning for Medical Image Classification. IEEE Transactions on Medical Imaging, 44(12), 4945-4959. https://doi.org/10.1109/tmi.2025.3584841

Vancouver

Du Y, Yu N, Wang S. Medical Knowledge Intervention Prompt Tuning for Medical Image Classification. IEEE Transactions on Medical Imaging. 2025;44(12):4945-4959. doi:10.1109/tmi.2025.3584841

BibTeX
@article{Du2025,
  author = {Du, Ye and Yu, Nanxi and Wang, Shujun},
  title = {Medical Knowledge Intervention Prompt Tuning for Medical Image Classification},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2025},
  volume = {44},
  number = {12},
  pages = {4945--4959},
  doi = {10.1109/tmi.2025.3584841}
}
Kostenlos starten →