A Secure, Scalable Large Language Model–Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study
Journal of Medical Internet Research · 2026
DOI: 10.2196/95780 ↗Kurz gefasst
Validierungsstudie in der Zeitschrift Journal of Medical Internet Research, 2026. Verfasst von Máté Posta und 3 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Molecular Life Sciences, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary. Hauptthemen laut MEDLINE: Data Mining, Information Storage and Retrieval. Die Arbeit nennt 63 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
A substantial proportion of clinically relevant information remains locked in unstructured narrative documents, creating a bottleneck for clinical research, biobank annotation, registry development, and real-world evidence generation. While large language models (LLMs) enable advanced clinical text mining, adoption is constrained by concerns regarding data security, multilingual performance, and reproducibility. Manual data abstraction remains predominant for registry curation and retrospective research, despite being labor intensive, costly, and prone to variability.
Objective
We developed and validated CIDER (Clinical Data Extractor), a secure, institutionally deployable, LLM-based pipeline for automated structured data extraction from routine clinical reports. We assessed the potential utility of the system for improving the completeness of clinical research datasets.
Methods
CIDER uses an asynchronous FastAPI-based architecture with a locally deployed vLLM inference engine running Qwen3-VL-32B-Instruct-FP8 model in an institution-controlled environment. The system was validated on 2073 real-world Hungarian-language histopathology reports (a challenging non-English setting), using a manually curated structured database as the reference standard. Seven variables were evaluated (sex, surgery year, T stage, N stage, organ, histology, and size). Extraction performance was assessed using exact-match accuracy, weighted F1-scores, Cohen κ statistics, and tolerance-based agreement thresholds for tumor size. Robustness was evaluated across temperatures from 0 to 2.0, and technical reproducibility was assessed at a temperature of 0.1 across 3 independent runs.
Results
The validation dataset comprised stand-alone native-text PDF pathology reports originating from multiple Hungarian oncology centers. Input document length showed a median of 3926 (mean 4258, SD 1057, IQR 3490-4688) tokens, while generated outputs contained a median of 63 (mean 62.5, SD 8.2, IQR 56-66) tokens. At a temperature of 0.1, CIDER achieved near-human agreement with expert-curated reference database, with exact-match accuracies of 99.5% for sex, 98.1% for surgery year, 95.8% for organ, 95.6% for T stage, 92.4% for N stage, 87.5% for histology, and 78.1% for tumor size. Weighted F1-scores ranged from 0.87 for histology to 0.995 for sex, while Cohen κ values ranged from 0.85 for N stage to 0.99 for sex. For tumor size extraction, 83.5% to 85.3% and 87.3% to 88.7% of predictions were within 5 mm and 10 mm of the manually curated values, respectively. CIDER additionally generated candidate extractions for variables omitted during manual curation, including 62.8% (713/1136) of missing T stages and 91.5% (289/316) of tumor size values. Sensitivity testing revealed high robustness, with negligible variance at temperature=0.1 and stable performance at high temperatures (temperature=2.0).
Conclusions
CIDER demonstrates that locally deployed open-weight LLMs can reliably extract structured clinical data from complex pathology reports while preserving institutional control over sensitive data. These findings support the feasibility of secure, institutionally deployable, LLM-based extraction systems for generating research-ready datasets, facilitating clinical registry development, improving dataset completeness, and enabling scalable reuse of unstructured clinical documentation.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Institute of Molecular Life Sciences, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary.
- Department of Bioinformatics, Semmelweis University, Budapest, Hungary.
- Institute of Transdisciplinary Discoveries, Medical School, University of Pécs, Pécs, Hungary.
ORCID-Profile: Máté Posta, Aida Figler, Zsófia Dobolyi, Balázs Győrffy
Zitieren
APA
Posta, M., Figler, A., Dobolyi, Z., & Győrffy, B. (2026). A Secure, Scalable Large Language Model–Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study. Journal of Medical Internet Research. https://doi.org/10.2196/95780
Vancouver
Posta M, Figler A, Dobolyi Z, Győrffy B. A Secure, Scalable Large Language Model–Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study. Journal of Medical Internet Research. 2026. doi:10.2196/95780
BibTeX
@article{Posta2026,
author = {Posta, Máté and Figler, Aida and Dobolyi, Zsófia and Győrffy, Balázs},
title = {A Secure, Scalable Large Language Model–Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study},
journal = {Journal of Medical Internet Research},
year = {2026},
doi = {10.2196/95780}
}
EUVIMED