EUVIMEDEUVIMED entdecken
Multizentrische StudieOpen Access

Enhancing Pulmonary Disease Prediction Using Large Language Models With Feature Summarization and Hybrid Retrieval-Augmented Generation: Multicenter Methodological Study Based on Radiology Report

Ronghao Li, Shuai Mao, Congmin Zhu, Yingliang Yang, Chunting Tan, Li Li, Xiangdong Mu, Honglei Liu, Yuqing Yang

Journal of Medical Internet Research · 11.06.2025 · 27 · e72638-e72638

DOI: 10.2196/72638 ↗

Kurz gefasst

Multizentrische Studie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 11.06.2025. Verfasst von Ronghao Li und 8 weiteren Autor:innen. Beteiligte Einrichtung: School of Biomedical Engineering, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing, 100069, China, 86 010-8391154…. Hauptthemen laut MEDLINE: Natural Language Processing, Lung Diseases. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The rapid advancements in natural language processing, particularly the development of large language models (LLMs), have opened new avenues for managing complex clinical text data. However, the inherent complexity and specificity of medical texts present significant challenges for the practical application of prompt engineering in diagnostic tasks.

Objective

This paper explores LLMs with new prompt engineering technology to enhance model interpretability and improve the prediction performance of pulmonary disease based on a traditional deep learning model.

Methods

A retrospective dataset including 2965 chest CT radiology reports was constructed. The reports were from 4 cohorts, namely, healthy individuals and patients with pulmonary tuberculosis, lung cancer, and pneumonia. Then, a novel prompt engineering strategy that integrates feature summarization (F-Sum), chain of thought (CoT) reasoning, and a hybrid retrieval-augmented generation (RAG) framework was proposed. A feature summarization approach, leveraging term frequency–inverse document frequency (TF-IDF) and K-means clustering, was used to extract and distill key radiological findings related to 3 diseases. Simultaneously, the hybrid RAG framework combined dense and sparse vector representations to enhance LLMs’ comprehension of disease-related text. In total, 3 state-of-the-art LLMs, GLM-4-Plus, GLM-4-air (Zhipu AI), and GPT-4o (OpenAI), were integrated with the prompt strategy to evaluate the efficiency in recognizing pneumonia, tuberculosis, and lung cancer. The traditional deep learning model, BERT (Bidirectional Encoder Representations from Transformers), was also compared to assess the superiority of LLMs. Finally, the proposed method was tested on an external validation dataset consisted of 343 chest computed tomography (CT) report from another hospital.

Results

Compared with BERT-based prediction model and various other prompt engineering techniques, our method with GLM-4-Plus achieved the best performance on test dataset, attaining an F 1 -score of 0.89 and accuracy of 0.89. On the external validation dataset, F 1 -score (0.86) and accuracy (0.92) of the proposed method with GPT-4o were the highest. Compared to the popular strategy with manually selected typical samples (few-shot) and CoT designed by doctors ( F 1 -score=0.83 and accuracy=0.83), the proposed method that summarized disease characteristics (F-Sum) based on LLM and automatically generated CoT performed better ( F 1 -score=0.89 and accuracy=0.90). Although the BERT-based model got similar results on the test dataset ( F 1 -score=0.85 and accuracy=0.88), its predictive performance significantly decreased on the external validation set ( F 1 -score=0.48 and accuracy=0.78).

Conclusions

These findings highlight the potential of LLMs to revolutionize pulmonary disease prediction, particularly in resource-constrained settings, by surpassing traditional models in both accuracy and flexibility. The proposed prompt engineering strategy not only improves predictive performance but also enhances the adaptability of LLMs in complex medical contexts, offering a promising tool for advancing disease diagnosis and clinical decision-making.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingLung DiseasesHumansRetrospective StudiesTomography, X-Ray ComputedDeep LearningLung NeoplasmsLarge Language Models

Stichwörter der Autor:innen

LLMRAGlarge language modelsprompt engineeringpulmonary disease predictionretrieval-augmented generation

Autor:innen und Institutionen

  • School of Biomedical Engineering, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing, 100069, China, 86 010-83911542.
  • State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China.
  • Department of Respiratory Medicine, Beijing Friendship Hospital, Capital Medical University, Beijing, China.
  • Beijing Respiratory and Critical Care Medicine Department, Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua University, Beijing, China.

Zitieren

APA

Li, R., Mao, S., Zhu, C., Yang, Y., Tan, C., Li, L., Mu, X., Liu, H., & Yang, Y. (2025). Enhancing Pulmonary Disease Prediction Using Large Language Models With Feature Summarization and Hybrid Retrieval-Augmented Generation: Multicenter Methodological Study Based on Radiology Report. Journal of Medical Internet Research, 27, e72638-e72638. https://doi.org/10.2196/72638

Vancouver

Li R, Mao S, Zhu C, Yang Y, Tan C, Li L, et al. Enhancing Pulmonary Disease Prediction Using Large Language Models With Feature Summarization and Hybrid Retrieval-Augmented Generation: Multicenter Methodological Study Based on Radiology Report. Journal of Medical Internet Research. 2025;27:e72638-e72638. doi:10.2196/72638

BibTeX
@article{Li2025,
  author = {Li, Ronghao and Mao, Shuai and Zhu, Congmin and Yang, Yingliang and Tan, Chunting and Li, Li and Mu, Xiangdong and Liu, Honglei and Yang, Yuqing},
  title = {Enhancing Pulmonary Disease Prediction Using Large Language Models With Feature Summarization and Hybrid Retrieval-Augmented Generation: Multicenter Methodological Study Based on Radiology Report},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e72638--e72638},
  doi = {10.2196/72638}
}
Kostenlos starten →