EUVIMEDEUVIMED entdecken
Crossref

Versatile Vision-Language Model for 3D Computed Tomography

Jiayu Lei, Ziqing Fan, Yanyong Zhang, Weidi Xie, Ya Zhang, Yanfeng Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (8) · 5945-5954

DOI: 10.1609/aaai.v40i8.37517 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Jiayu Lei und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Representation learning serves as a foundational component of medical vision-language models (MVLMs), enabling cross-modal alignment, semantic consistency, and enhanced generalization capabilities for downstream tasks. As generalist models rapidly evolve, there is a pressing need to unify diverse downstream tasks, such as diagnosis, segmentation, report generation, and multiple choice within a cohesive framework, demanding more efficient and versatile visual representation learning. However, current MVLMs predominately follow CLIP-style vision pretraining, failing to leverage heterogeneous data resources with multi-dimensional imaging and diverse annotation forms. And there lacks systematic analysis of efficient vision encoder design across varied downstream applications, including diagnosis, segmentation, and text generation tasks, particularly for volumetric imaging like Computed Tomography (CT). Besides, current MVLMs exhibit constrained voxel-level capabilities, lacking effective multi-task instruction tuning framework capable of achieving robust performance across various downstream tasks. To address these challenges, we propose CTInstruct, a novel MVLM employing a hybrid ResNet-ViT encoder with multi-granular vision-language pretraining for efficient heterogeneous data modeling, and unified instruction tuning that jointly optimizes discriminative, generative, and voxel-level reasoning for volumetric medical imaging. CTInstruct achieves SOTA performance across 8 CT benchmarks, setting a new standard for data-efficient multimodal learning in medical imaging.

Zitieren

APA

Lei, J., Fan, Z., Zhang, Y., Xie, W., Zhang, Y., & Wang, Y. (2026). Versatile Vision-Language Model for 3D Computed Tomography. Proceedings of the AAAI Conference on Artificial Intelligence, 40(8), 5945-5954. https://doi.org/10.1609/aaai.v40i8.37517

Vancouver

Lei J, Fan Z, Zhang Y, Xie W, Zhang Y, Wang Y. Versatile Vision-Language Model for 3D Computed Tomography. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(8):5945-5954. doi:10.1609/aaai.v40i8.37517

BibTeX
@article{Lei2026,
  author = {Lei, Jiayu and Fan, Ziqing and Zhang, Yanyong and Xie, Weidi and Zhang, Ya and Wang, Yanfeng},
  title = {Versatile Vision-Language Model for 3D Computed Tomography},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {8},
  pages = {5945--5954},
  doi = {10.1609/aaai.v40i8.37517}
}
Kostenlos starten →