EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CXR-LLaVA: a multimodal large language model for interpreting chest X-ray images

Seowoo Lee, Jiwon Youn, Hyungjin Kim, Mansu Kim, Soon Ho Yoon

European Radiology · 15.01.2025 · 35 (7) · 4374-4386

DOI: 10.1007/s00330-024-11339-6 ↗

Kurz gefasst

Fachartikel in der Zeitschrift European Radiology, online erschienen am 15.01.2025. Verfasst von Seowoo Lee und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, Seoul, Republic of Korea. Hauptthemen laut MEDLINE: Radiography, Thoracic, Radiographic Image Interpretation, Computer-Assisted. Die Arbeit wurde bisher 83-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation of Korea, Institute of Information and Communications Technology Planning and Evaluation (IITP), Korean Government, NAVER Digital Bio Innovation Research Fund.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

This study aimed to develop an open-source multimodal large language model (CXR-LLaVA) for interpreting chest X-ray images (CXRs), leveraging recent advances in large language models (LLMs) to potentially replicate the image interpretation skills of human radiologists. Materials and methods For training, we collected 592,580 publicly available CXRs, of which 374,881 had labels for certain radiographic abnormalities (Dataset 1) and 217,699 provided free-text radiology reports (Dataset 2). After pre-training a vision transformer with Dataset 1, we integrated it with an LLM influenced by the LLaVA network. Then, the model was fine-tuned, primarily using Dataset 2. The model’s diagnostic performance for major pathological findings was evaluated, along with the acceptability of radiologic reports by human radiologists, to gauge its potential for autonomous reporting.

Results

The model demonstrated impressive performance in test sets, achieving an average F1 score of 0.81 for six major pathological findings in the MIMIC internal test set and 0.56 for six major pathological findings in the external test set. The model’s F1 scores surpassed those of GPT-4-vision and Gemini-Pro-Vision in both test sets. In human radiologist evaluations of the external test set, the model achieved a 72.7% success rate in autonomous reporting, slightly below the 84.0% rate of ground truth reports.

Conclusion

This study highlights the significant potential of multimodal LLMs for CXR interpretation, while also acknowledging the performance limitations. Despite these challenges, we believe that making our model open-source will catalyze further research, expanding its effectiveness and applicability in various clinical contexts. Key Points Question How can a multimodal large language model be adapted to interpret chest X-rays and generate radiologic reports?

Findings

The developed CXR-LLaVA model effectively detects major pathological findings in chest X-rays and generates radiologic reports with a higher accuracy compared to general-purpose models. Clinical relevance This study demonstrates the potential of multimodal large language models to support radiologists by autonomously generating chest X-ray reports, potentially reducing diagnostic workloads and improving radiologist efficiency.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Radiography, ThoracicRadiographic Image Interpretation, Computer-AssistedHumansLarge Language Models

Stichwörter der Autor:innen

Deep learningImage interpretation, Image interpretation (computer-assisted)Radiography (thoracic)Thorax

Autor:innen und Institutionen

  • Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, Seoul, Republic of Korea.
  • AI Graduate School, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea.
  • AI Graduate School, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea. mansu.kim@gist.ac.kr.
  • Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, Seoul, Republic of Korea. yshoka@snu.ac.kr.

Förderung

  • National Research Foundation of Korea
  • Institute of Information and Communications Technology Planning and Evaluation (IITP), Korean Government
  • NAVER Digital Bio Innovation Research Fund

Zitieren

APA

Lee, S., Youn, J., Kim, H., Kim, M., & Yoon, S. H. (2025). CXR-LLaVA: a multimodal large language model for interpreting chest X-ray images. European Radiology, 35(7), 4374-4386. https://doi.org/10.1007/s00330-024-11339-6

Vancouver

Lee S, Youn J, Kim H, Kim M, Yoon SH. CXR-LLaVA: a multimodal large language model for interpreting chest X-ray images. European Radiology. 2025;35(7):4374-4386. doi:10.1007/s00330-024-11339-6

BibTeX
@article{Lee2025,
  author = {Lee, Seowoo and Youn, Jiwon and Kim, Hyungjin and Kim, Mansu and Yoon, Soon Ho},
  title = {CXR-LLaVA: a multimodal large language model for interpreting chest X-ray images},
  journal = {European Radiology},
  year = {2025},
  volume = {35},
  number = {7},
  pages = {4374--4386},
  doi = {10.1007/s00330-024-11339-6}
}
Kostenlos starten →