EUVIMEDEUVIMED entdecken
Crossref

AMVLM: Alignment-Multiplicity Aware Vision-Language Model for Semi-Supervised Medical Image Segmentation

Qingtao Pan, Zhengrong Li, Wenhao Qiao, Jingjiao Lou, Qing Yang, Guang Yang, Bing Ji

IEEE Transactions on Medical Imaging · 2025-11 · 44 (11) · 4307-4322

DOI: 10.1109/tmi.2025.3573018 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2025-11. Verfasst von Qingtao Pan und 6 weiteren Autor:innen. Hauptthemen laut MEDLINE: Image Processing, Computer-Assisted, Supervised Machine Learning. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: European Research Council, Medical Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Low-quality pseudo labels pose a significant obstacle in semi-supervised medical image segmentation (SSMIS), impeding consistency learning on unlabeled data. Leveraging vision-language model (VLM) holds promise in ameliorating pseudo label quality by employing textual prompts to delineate segmentation regions, but it faces the challenge of cross-modal alignment uncertainty due to multiple correspondences (multiple images/texts tend to correspond to one text/image). Existing VLMs address this challenge by modeling semantics as distributions but such distributions lead to semantic degradation. To address these problems, we propose Alignment-Multiplicity Aware Vision-Language Model (AMVLM), a new VLM pretraining paradigm with two novel similarity metric strategies. (i) Cross-modal Similarity Supervision (CSS) proposes a probability distribution transformer to supervise similarity scores across fine-granularity semantics through measuring cross-modal distribution disparities, thus learning cross-modal multiple alignments. (ii) Intra-modal Contrastive Learning (ICL) takes into account the similarity metric of coarse-fine granularity information within each modality to encourage cross-modal semantic consistency. Furthermore, using the pretrained AMVLM, we propose a pioneering text-guided SSMIS network to compensate for the quality deficiencies of pseudo-labels. This network incorporates a text mask generator to produce multimodal supervision information, enhancing pseudo label quality and the model's consistency learning. Extensive experimentation validates the efficacy of our AMVLM-driven SSMIS, showcasing superior performance across four publicly available datasets. The code will be available at: https://github.com/QingtaoPan/AMVLM.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Image Processing, Computer-AssistedSupervised Machine LearningHumansAlgorithmsDatabases, Factual

Förderung

  • European Research Council
  • Medical Research Council

Zitieren

APA

Pan, Q., Li, Z., Qiao, W., Lou, J., Yang, Q., Yang, G., & Ji, B. (2025). AMVLM: Alignment-Multiplicity Aware Vision-Language Model for Semi-Supervised Medical Image Segmentation. IEEE Transactions on Medical Imaging, 44(11), 4307-4322. https://doi.org/10.1109/tmi.2025.3573018

Vancouver

Pan Q, Li Z, Qiao W, Lou J, Yang Q, Yang G, et al. AMVLM: Alignment-Multiplicity Aware Vision-Language Model for Semi-Supervised Medical Image Segmentation. IEEE Transactions on Medical Imaging. 2025;44(11):4307-4322. doi:10.1109/tmi.2025.3573018

BibTeX
@article{Pan2025,
  author = {Pan, Qingtao and Li, Zhengrong and Qiao, Wenhao and Lou, Jingjiao and Yang, Qing and Yang, Guang and Ji, Bing},
  title = {AMVLM: Alignment-Multiplicity Aware Vision-Language Model for Semi-Supervised Medical Image Segmentation},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2025},
  volume = {44},
  number = {11},
  pages = {4307--4322},
  doi = {10.1109/tmi.2025.3573018}
}
Kostenlos starten →