Patient-specific multimodal learning with multi-view contrastive alignment for chest X-ray report generation
Bioinformatics · 03.08.2026 · 42 (8)
DOI: 10.1093/bioinformatics/btag566 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioinformatics, online erschienen am 28.07.2026. Verfasst von Qiguang Miao und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Technology, Xidian University, Xi'an, Shaanxi 710071, China. Hauptthemen laut MEDLINE: Radiography, Thoracic, Machine Learning. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the National Natural Science Foundations of China, Provincial Key Research and Development Program of Shaanxi, China Education Society of Electronics Teaching Reform.
Abstract (Originalsprache)
Motivation Radiology reports play a pivotal role in guiding treatment planning and enabling effective doctor-patient communication. However, their manual composition imposes a substantial workload on radiologists. Although automatic radiology report generation has emerged as a promising alternative, existing approaches predominantly rely on single-view chest X-rays and fail to adequately leverage patient-specific context, thereby limiting diagnostic accuracy. Results To address this challenge, we propose EVOKE, a novel chest X-ray report generation framework that incorporates multi-view contrastive learning and patient-specific knowledge. Specifically, we introduce a multi-view contrastive learning method that captures semantic correspondences both among multi-view radiographs within a study and between these radiographs and their associated report, thereby improving visual representation learning. We further present a knowledge-guided report generation module that integrates available patient-specific knowledge (i.e. indication, which includes symptom descriptions) to facilitate the generation of accurate and coherent radiology reports. To support research in multi-view report generation, we construct Multi-view CXR and Two-view CXR datasets using publicly available sources. Our proposed EVOKE surpasses recent state-of-the-art methods across multiple datasets, achieving a 2.9% F1 RadGraph improvement on MIMIC-CXR, a 5.0% BLEU-1 improvement on MIMIC-ABN, a 1.5% BLEU-4 improvement on Multi-view CXR, and an 8.2% F1,mic-14 CheXbert improvement on Two-view CXR. Availability Code is publicly available at https://github.com/mk-runner/EVOKE, with an archived release available on Zenodo (doi:10.5281/zenodo.21000219).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- School of Computer Science and Technology, Xidian University, Xi'an, Shaanxi 710071, China.
- Xi'an Key Laboratory of Big Data and Intelligent Vision, Xi'an, Shaanxi 710071, China.
- Key Laboratory of Smart Education of Shaanxi Higher Education Institutes, Xidian University, Xi'an, Shaanxi 710071, China.
- School of Medicine, Shanghai Jiao Tong University, Shanghai 200025, China.
- Department of Neurology, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang 150001, China.
ORCID-Profile: Qiguang Miao, Kang Liu, Zhuoqi Ma, Yunan Li, Xiaolu Kang, Kun Xie
Förderung
- the National Natural Science Foundations of China
- Provincial Key Research and Development Program of Shaanxi
- China Education Society of Electronics Teaching Reform
- Shaanxi Higher Education Teaching Reform Research
Zitieren
APA
Miao, Q., Liu, K., Ma, Z., Li, Y., Kang, X., Liu, R., Liu, T., & Xie, K. (2026). Patient-specific multimodal learning with multi-view contrastive alignment for chest X-ray report generation. Bioinformatics, 42(8). https://doi.org/10.1093/bioinformatics/btag566
Vancouver
Miao Q, Liu K, Ma Z, Li Y, Kang X, Liu R, et al. Patient-specific multimodal learning with multi-view contrastive alignment for chest X-ray report generation. Bioinformatics. 2026;42(8). doi:10.1093/bioinformatics/btag566
BibTeX
@article{Miao2026,
author = {Miao, Qiguang and Liu, Kang and Ma, Zhuoqi and Li, Yunan and Kang, Xiaolu and Liu, Ruixuan and Liu, Tianyi and Xie, Kun},
title = {Patient-specific multimodal learning with multi-view contrastive alignment for chest X-ray report generation},
journal = {Bioinformatics},
year = {2026},
volume = {42},
number = {8},
doi = {10.1093/bioinformatics/btag566}
}
EUVIMED