EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Patient-specific multimodal learning with multi-view contrastive alignment for chest X-ray report generation

Qiguang Miao iD, Kang Liu iD, Zhuoqi Ma iD, Yunan Li iD, Xiaolu Kang iD, Ruixuan Liu, Tianyi Liu, Kun Xie iD

Bioinformatics · 03.08.2026 · 42 (8)

DOI: 10.1093/bioinformatics/btag566 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics, online erschienen am 28.07.2026. Verfasst von Qiguang Miao und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Technology, Xidian University, Xi'an, Shaanxi 710071, China. Hauptthemen laut MEDLINE: Radiography, Thoracic, Machine Learning. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the National Natural Science Foundations of China, Provincial Key Research and Development Program of Shaanxi, China Education Society of Electronics Teaching Reform.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation Radiology reports play a pivotal role in guiding treatment planning and enabling effective doctor-patient communication. However, their manual composition imposes a substantial workload on radiologists. Although automatic radiology report generation has emerged as a promising alternative, existing approaches predominantly rely on single-view chest X-rays and fail to adequately leverage patient-specific context, thereby limiting diagnostic accuracy. Results To address this challenge, we propose EVOKE, a novel chest X-ray report generation framework that incorporates multi-view contrastive learning and patient-specific knowledge. Specifically, we introduce a multi-view contrastive learning method that captures semantic correspondences both among multi-view radiographs within a study and between these radiographs and their associated report, thereby improving visual representation learning. We further present a knowledge-guided report generation module that integrates available patient-specific knowledge (i.e. indication, which includes symptom descriptions) to facilitate the generation of accurate and coherent radiology reports. To support research in multi-view report generation, we construct Multi-view CXR and Two-view CXR datasets using publicly available sources. Our proposed EVOKE surpasses recent state-of-the-art methods across multiple datasets, achieving a 2.9% F1 RadGraph improvement on MIMIC-CXR, a 5.0% BLEU-1 improvement on MIMIC-ABN, a 1.5% BLEU-4 improvement on Multi-view CXR, and an 8.2% F1,mic-14 CheXbert improvement on Two-view CXR. Availability Code is publicly available at https://github.com/mk-runner/EVOKE, with an archived release available on Zenodo (doi:10.5281/zenodo.21000219).

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Radiography, ThoracicMachine LearningHumansAlgorithms

Autor:innen und Institutionen

  • School of Computer Science and Technology, Xidian University, Xi'an, Shaanxi 710071, China.
  • Xi'an Key Laboratory of Big Data and Intelligent Vision, Xi'an, Shaanxi 710071, China.
  • Key Laboratory of Smart Education of Shaanxi Higher Education Institutes, Xidian University, Xi'an, Shaanxi 710071, China.
  • School of Medicine, Shanghai Jiao Tong University, Shanghai 200025, China.
  • Department of Neurology, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang 150001, China.

ORCID-Profile: Qiguang Miao, Kang Liu, Zhuoqi Ma, Yunan Li, Xiaolu Kang, Kun Xie

Förderung

  • the National Natural Science Foundations of China
  • Provincial Key Research and Development Program of Shaanxi
  • China Education Society of Electronics Teaching Reform
  • Shaanxi Higher Education Teaching Reform Research

Zitieren

APA

Miao, Q., Liu, K., Ma, Z., Li, Y., Kang, X., Liu, R., Liu, T., & Xie, K. (2026). Patient-specific multimodal learning with multi-view contrastive alignment for chest X-ray report generation. Bioinformatics, 42(8). https://doi.org/10.1093/bioinformatics/btag566

Vancouver

Miao Q, Liu K, Ma Z, Li Y, Kang X, Liu R, et al. Patient-specific multimodal learning with multi-view contrastive alignment for chest X-ray report generation. Bioinformatics. 2026;42(8). doi:10.1093/bioinformatics/btag566

BibTeX
@article{Miao2026,
  author = {Miao, Qiguang and Liu, Kang and Ma, Zhuoqi and Li, Yunan and Kang, Xiaolu and Liu, Ruixuan and Liu, Tianyi and Xie, Kun},
  title = {Patient-specific multimodal learning with multi-view contrastive alignment for chest X-ray report generation},
  journal = {Bioinformatics},
  year = {2026},
  volume = {42},
  number = {8},
  doi = {10.1093/bioinformatics/btag566}
}
Kostenlos starten →