EUVIMEDEUVIMED entdecken
Open Access

An Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study

Lei Wang iD, Yinyao Ma iD, Wenshuai Bi iD, Hanlin Lv iD, Yuxiang Li iD

Journal of Medical Internet Research · 29.03.2024 · 26 · e54580

DOI: 10.2196/54580 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Medical Internet Research, online erschienen am 29.03.2024. Verfasst von Lei Wang und 4 weiteren Autor:innen. Beteiligte Einrichtung: BGI Research, Wuhan, China. Hauptthemen laut MEDLINE: Electronic Health Records, Data Mining. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The study of disease progression relies on clinical data, including text data, and extracting valuable features from text data has been a research hot spot. With the rise of large language models (LLMs), semantic-based extraction pipelines are gaining acceptance in clinical research. However, the security and feature hallucination issues of LLMs require further attention.

Objective

This study aimed to introduce a novel modular LLM pipeline, which could semantically extract features from textual patient admission records.

Methods

The pipeline was designed to process a systematic succession of concept extraction, aggregation, question generation, corpus extraction, and question-and-answer scale extraction, which was tested via 2 low-parameter LLMs: Qwen-14B-Chat (QWEN) and Baichuan2-13B-Chat (BAICHUAN). A data set of 25,709 pregnancy cases from the People’s Hospital of Guangxi Zhuang Autonomous Region, China, was used for evaluation with the help of a local expert’s annotation. The pipeline was evaluated with the metrics of accuracy and precision, null ratio, and time consumption. Additionally, we evaluated its performance via a quantified version of Qwen-14B-Chat on a consumer-grade GPU.

Results

The pipeline demonstrates a high level of precision in feature extraction, as evidenced by the accuracy and precision results of Qwen-14B-Chat (95.52% and 92.93%, respectively) and Baichuan2-13B-Chat (95.86% and 90.08%, respectively). Furthermore, the pipeline exhibited low null ratios and variable time consumption. The INT4-quantified version of QWEN delivered an enhanced performance with 97.28% accuracy and a 0% null ratio.

Conclusions

The pipeline exhibited consistent performance across different LLMs and efficiently extracted clinical features from textual data. It also showed reliable performance on consumer-grade hardware. This approach offers a viable and effective solution for mining clinical research data from textual records.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Electronic Health RecordsData MiningHumansNatural Language ProcessingChinaLanguage

Stichwörter der Autor:innen

clinical data extractionfeature hallucinationlarge language modelsmodular approachunstructured data processing

Autor:innen und Institutionen

  • BGI Research, Wuhan, China.
  • Guangdong Bigdata Engineering Technology Research Center for Life Sciences, BGI Research, Shenzhen, China.
  • Department of Obstetrics, People's Hospital of Guangxi Zhuang Autonomous Region, Nanning, China.

ORCID-Profile: Lei Wang, Yinyao Ma, Wenshuai Bi, Hanlin Lv, Yuxiang Li

Zitieren

APA

Wang, L., Ma, Y., Bi, W., Lv, H., & Li, Y. (2024). An Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study. Journal of Medical Internet Research, 26, e54580. https://doi.org/10.2196/54580

Vancouver

Wang L, Ma Y, Bi W, Lv H, Li Y. An Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study. Journal of Medical Internet Research. 2024;26:e54580. doi:10.2196/54580

BibTeX
@article{Wang2024,
  author = {Wang, Lei and Ma, Yinyao and Bi, Wenshuai and Lv, Hanlin and Li, Yuxiang},
  title = {An Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  volume = {26},
  pages = {e54580},
  doi = {10.2196/54580}
}
Kostenlos starten →