LIGHT: Learning Image-Text Grounding for Hierarchical Tumor Localization and Subtype Classification in Multimodal Brain MRI
IEEE Transactions on Medical Imaging · 2026-11 · 45 (11) · 5265-5275
DOI: 10.1109/tmi.2026.3718445 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2026-11. Verfasst von Zhaoyu Qiu und 6 weiteren Autor:innen. Hauptthemen laut MEDLINE: Brain Neoplasms, Magnetic Resonance Imaging, Image Interpretation, Computer-Assisted. Gefördert von: National Natural Science Foundation of China, China Ministry of Science and Technology, Key Research and Development Program of Guangdong Province, China.
Abstract (Originalsprache)
Accurate MRI-based brain tumor analysis requires not only tumor subtype classification but also localization at an anatomical granularity that is consistent with radiology reports. Most vision-only methods address localization and classification as separate label-prediction tasks, and therefore provide limited alignment with the fine-grained anatomical semantics used in routine reporting. To address this limitation, we propose LIGHT (Learning Image-text Grounding for Hierarchical Tumor analysis), a 3D vision-language framework that formulates brain tumor localization and subtype classification as image-text retrieval in a shared embedding space. Given multimodal MRI inputs, LIGHT retrieves coarse anatomical regions, fine-grained subregions, and tumor subtypes in a unified coarse-to-fine procedure. The framework has three main components: 1) a hierarchical retrieval vocabulary containing 21 coarse-grained regions, 563 fine-grained subregions, and 5 tumor subtypes; 2) large-scale foundation pretraining on a curated patient-disjoint in-house dataset of 99,813 MRI-report pairs; and 3) grounded task fine-tuning that uses segmentation-guided tumor crops and multi-template prompt supervision to improve tumor-prompt alignment. Across 11,034 annotated 3D brain tumor MRI cases from in-house, external clinical, and public datasets, LIGHT achieved 72.1% accuracy for coarse-grained localization, 70.1% top-1 accuracy for fine-grained localization, and 76.0%, 83.7%, and 84.6% accuracy for subtype classification on three 3D datasets, respectively. Additional public 2D subtype-classification experiments provide further evidence of cross-setting subtype recognition. These results suggest that report-grounded image-text retrieval can provide interpretable anatomical and diagnostic outputs for brain tumor MRI analysis. Code: https://github.com/qiuzhaoyu/LIGHT.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- National Natural Science Foundation of China
- China Ministry of Science and Technology
- Key Research and Development Program of Guangdong Province, China
- National Key Research and Development Program of China
- High Performance Computing (HPC) Platform of ShanghaiTech University.
Zitieren
APA
Qiu, Z., Liu, Y., Li, J., Jin, L., Ding, Z., Shi, F., & Shen, D. (2026). LIGHT: Learning Image-Text Grounding for Hierarchical Tumor Localization and Subtype Classification in Multimodal Brain MRI. IEEE Transactions on Medical Imaging, 45(11), 5265-5275. https://doi.org/10.1109/tmi.2026.3718445
Vancouver
Qiu Z, Liu Y, Li J, Jin L, Ding Z, Shi F, et al. LIGHT: Learning Image-Text Grounding for Hierarchical Tumor Localization and Subtype Classification in Multimodal Brain MRI. IEEE Transactions on Medical Imaging. 2026;45(11):5265-5275. doi:10.1109/tmi.2026.3718445
BibTeX
@article{Qiu2026,
author = {Qiu, Zhaoyu and Liu, Yuxiao and Li, Jianrui and Jin, Lei and Ding, Zhongxiang and Shi, Feng and Shen, Dinggang},
title = {LIGHT: Learning Image-Text Grounding for Hierarchical Tumor Localization and Subtype Classification in Multimodal Brain MRI},
journal = {IEEE Transactions on Medical Imaging},
year = {2026},
volume = {45},
number = {11},
pages = {5265--5275},
doi = {10.1109/tmi.2026.3718445}
}
EUVIMED