EUVIMEDEUVIMED entdecken
Crossref

LIGHT: Learning Image-Text Grounding for Hierarchical Tumor Localization and Subtype Classification in Multimodal Brain MRI

Zhaoyu Qiu, Yuxiao Liu, Jianrui Li, Lei Jin, Zhongxiang Ding, Feng Shi, Dinggang Shen

IEEE Transactions on Medical Imaging · 2026-11 · 45 (11) · 5265-5275

DOI: 10.1109/tmi.2026.3718445 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2026-11. Verfasst von Zhaoyu Qiu und 6 weiteren Autor:innen. Hauptthemen laut MEDLINE: Brain Neoplasms, Magnetic Resonance Imaging, Image Interpretation, Computer-Assisted. Gefördert von: National Natural Science Foundation of China, China Ministry of Science and Technology, Key Research and Development Program of Guangdong Province, China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurate MRI-based brain tumor analysis requires not only tumor subtype classification but also localization at an anatomical granularity that is consistent with radiology reports. Most vision-only methods address localization and classification as separate label-prediction tasks, and therefore provide limited alignment with the fine-grained anatomical semantics used in routine reporting. To address this limitation, we propose LIGHT (Learning Image-text Grounding for Hierarchical Tumor analysis), a 3D vision-language framework that formulates brain tumor localization and subtype classification as image-text retrieval in a shared embedding space. Given multimodal MRI inputs, LIGHT retrieves coarse anatomical regions, fine-grained subregions, and tumor subtypes in a unified coarse-to-fine procedure. The framework has three main components: 1) a hierarchical retrieval vocabulary containing 21 coarse-grained regions, 563 fine-grained subregions, and 5 tumor subtypes; 2) large-scale foundation pretraining on a curated patient-disjoint in-house dataset of 99,813 MRI-report pairs; and 3) grounded task fine-tuning that uses segmentation-guided tumor crops and multi-template prompt supervision to improve tumor-prompt alignment. Across 11,034 annotated 3D brain tumor MRI cases from in-house, external clinical, and public datasets, LIGHT achieved 72.1% accuracy for coarse-grained localization, 70.1% top-1 accuracy for fine-grained localization, and 76.0%, 83.7%, and 84.6% accuracy for subtype classification on three 3D datasets, respectively. Additional public 2D subtype-classification experiments provide further evidence of cross-setting subtype recognition. These results suggest that report-grounded image-text retrieval can provide interpretable anatomical and diagnostic outputs for brain tumor MRI analysis. Code: https://github.com/qiuzhaoyu/LIGHT.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Brain NeoplasmsMagnetic Resonance ImagingImage Interpretation, Computer-AssistedHumansBrainMultimodal ImagingAlgorithmsMachine Learning

Förderung

  • National Natural Science Foundation of China
  • China Ministry of Science and Technology
  • Key Research and Development Program of Guangdong Province, China
  • National Key Research and Development Program of China
  • High Performance Computing (HPC) Platform of ShanghaiTech University.

Zitieren

APA

Qiu, Z., Liu, Y., Li, J., Jin, L., Ding, Z., Shi, F., & Shen, D. (2026). LIGHT: Learning Image-Text Grounding for Hierarchical Tumor Localization and Subtype Classification in Multimodal Brain MRI. IEEE Transactions on Medical Imaging, 45(11), 5265-5275. https://doi.org/10.1109/tmi.2026.3718445

Vancouver

Qiu Z, Liu Y, Li J, Jin L, Ding Z, Shi F, et al. LIGHT: Learning Image-Text Grounding for Hierarchical Tumor Localization and Subtype Classification in Multimodal Brain MRI. IEEE Transactions on Medical Imaging. 2026;45(11):5265-5275. doi:10.1109/tmi.2026.3718445

BibTeX
@article{Qiu2026,
  author = {Qiu, Zhaoyu and Liu, Yuxiao and Li, Jianrui and Jin, Lei and Ding, Zhongxiang and Shi, Feng and Shen, Dinggang},
  title = {LIGHT: Learning Image-Text Grounding for Hierarchical Tumor Localization and Subtype Classification in Multimodal Brain MRI},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2026},
  volume = {45},
  number = {11},
  pages = {5265--5275},
  doi = {10.1109/tmi.2026.3718445}
}
Kostenlos starten →