An Entity Extraction pipeline for Medical Text Records Utilizing Large Language Models: An Analytical Study (Preprint)
Greater South Information System · 15.11.2023
DOI: 10.60692/vg4n4-p0262 ↗Kurz gefasst
Fachartikel in der Zeitschrift Greater South Information System, erschienen am 15.11.2023. Verfasst von Lei Wang und 4 weiteren Autor:innen.
Abstract (Originalsprache)
The study of disease progression relies on clinical data, including text data, and extracting valuable features from text data has been a research hot spot. With the rise of large language models (LLMs), semantic-based extraction pipelines are gaining acceptance in clinical research. However, the security and feature hallucination issues of LLMs require further attention.This study aimed to introduce a novel modular LLM pipeline, which could semantically extract features from textual patient admission records.The pipeline was designed to process a systematic succession of concept extraction, aggregation, question generation, corpus extraction, and question-and-answer scale extraction, which was tested via 2 low-parameter LLMs: Qwen-14B-Chat (QWEN) and Baichuan2-13B-Chat (BAICHUAN). A data set of 25,709 pregnancy cases from the People's Hospital of Guangxi Zhuang Autonomous Region, China, was used for evaluation with the help of a local expert's annotation. The pipeline was evaluated with the metrics of accuracy and precision, null ratio, and time consumption. Additionally, we evaluated its performance via a quantified version of Qwen-14B-Chat on a consumer-grade GPU.The pipeline demonstrates a high level of precision in feature extraction, as evidenced by the accuracy and precision results of Qwen-14B-Chat (95.52% and 92.93%, respectively) and Baichuan2-13B-Chat (95.86% and 90.08%, respectively). Furthermore, the pipeline exhibited low null ratios and variable time consumption. The INT4-quantified version of QWEN delivered an enhanced performance with 97.28% accuracy and a 0% null ratio.The pipeline exhibited consistent performance across different LLMs and efficiently extracted clinical features from textual data. It also showed reliable performance on consumer-grade hardware. This approach offers a viable and effective solution for mining clinical research data from textual records.
Schlagwörter
Zitieren
APA
Wang, L., Ma, Y., Bi, W. L., Lv, H. Y., & Li, Y. (2023). An Entity Extraction pipeline for Medical Text Records Utilizing Large Language Models: An Analytical Study (Preprint). Greater South Information System. https://doi.org/10.60692/vg4n4-p0262
Vancouver
Wang L, Ma Y, Bi WL, Lv HY, Li Y. An Entity Extraction pipeline for Medical Text Records Utilizing Large Language Models: An Analytical Study (Preprint). Greater South Information System. 2023. doi:10.60692/vg4n4-p0262
BibTeX
@article{Wang2023,
author = {Wang, Lei and Ma, Yuanliang and Bi, Wenya Linda and Lv, Hong Yan and Li, Yuxiang},
title = {An Entity Extraction pipeline for Medical Text Records Utilizing Large Language Models: An Analytical Study (Preprint)},
journal = {Greater South Information System},
year = {2023},
doi = {10.60692/vg4n4-p0262}
}
EUVIMED