EUVIMEDEUVIMED entdecken
Open Access

Knowledge-enhanced medical image classification via descriptive priors from large language models

Yuhang Zhang, Yiming Xu, Peilin Chen, Shiqi Wang, Qi Song iD, Lei Yu, Wei Cai

Health Information Science and Systems · 28.09.2025 · 13 (1)

DOI: 10.1007/s13755-025-00380-4 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Health Information Science and Systems, online erschienen am 28.09.2025. Verfasst von Yuhang Zhang und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Technology, University of Science and Technology of China, Hefei, 230001 Anhui China. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: The Plans for Major Provincial Science&Technology Projects, the Joint Fund for Medical Artificial Intelligence, the Anhui Provincial Natural Science Foundation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Medical image classification aims to categorise clinically significant imaging patterns, thereby facilitating accurate and timely diagnosis. However, existing approaches predominantly rely on visual features extracted from raw pixel data, often overlooking fine-grained diagnostic cues grounded in medical expertise. To address this limitation, we propose a novel knowledge-enhanced Model, called KEM, that leverages medical large vision-language models (Medical LVLMs) as domain experts to generate descriptive priors, which are used to guide and support clinical decision-making. Specifically, we prompt Medical LVLMs to generate rich, Multi-Dimension clinical descriptions tailored to each input image, capturing nuanced semantics. These descriptive priors are then encoded and fused with visual features through a dual cross-attention module, which enables bidirectional interaction and alignment between modalities. This design allows the model to dynamically attend to both textual and visual cues, thereby enhancing its ability to recognise subtle disease patterns. Comprehensive experiments on four benchmark datasets demonstrate that the proposed method significantly outperforms state-of-the-art vision-only models and exhibits strong generalization across varied clinical settings.

Schlagwörter

Stichwörter der Autor:innen

Descriptive PriorsDual Cross-Attention ModuleKnowledge-Enhanced ModelMedical Image ClassificationMedical Large Language Models

Autor:innen und Institutionen

  • School of Computer Science and Technology, University of Science and Technology of China, Hefei, 230001 Anhui China.
  • Department of Computer Science, City University of Hong Kong, Kowloon, 999077 Hong Kong China.
  • School of Medical Information Technology, Anhui University of Chinese Medicine, Hefei, 230012 Anhui China.
  • The Hepatobiliary Surgery of The First Affiliated Hospital of USTC, University of Science and Technology of China, Hefei, 230001 Anhui China.

ORCID-Profile: Qi Song

Förderung

  • The Plans for Major Provincial Science&Technology Projects
  • the Joint Fund for Medical Artificial Intelligence
  • the Anhui Provincial Natural Science Foundation

Zitieren

APA

Zhang, Y., Xu, Y., Chen, P., Wang, S., Song, Q., Yu, L., & Cai, W. (2025). Knowledge-enhanced medical image classification via descriptive priors from large language models. Health Information Science and Systems, 13(1). https://doi.org/10.1007/s13755-025-00380-4

Vancouver

Zhang Y, Xu Y, Chen P, Wang S, Song Q, Yu L, et al. Knowledge-enhanced medical image classification via descriptive priors from large language models. Health Information Science and Systems. 2025;13(1). doi:10.1007/s13755-025-00380-4

BibTeX
@article{Zhang2025,
  author = {Zhang, Yuhang and Xu, Yiming and Chen, Peilin and Wang, Shiqi and Song, Qi and Yu, Lei and Cai, Wei},
  title = {Knowledge-enhanced medical image classification via descriptive priors from large language models},
  journal = {Health Information Science and Systems},
  year = {2025},
  volume = {13},
  number = {1},
  doi = {10.1007/s13755-025-00380-4}
}
Kostenlos starten →