EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access

A Secure, Scalable Large Language Model–Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study

Máté Posta iD, Aida Figler iD, Zsófia Dobolyi iD, Balázs Győrffy iD

Journal of Medical Internet Research · 2026

DOI: 10.2196/95780 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift Journal of Medical Internet Research, 2026. Verfasst von Máté Posta und 3 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Molecular Life Sciences, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary. Hauptthemen laut MEDLINE: Data Mining, Information Storage and Retrieval. Die Arbeit nennt 63 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

A substantial proportion of clinically relevant information remains locked in unstructured narrative documents, creating a bottleneck for clinical research, biobank annotation, registry development, and real-world evidence generation. While large language models (LLMs) enable advanced clinical text mining, adoption is constrained by concerns regarding data security, multilingual performance, and reproducibility. Manual data abstraction remains predominant for registry curation and retrospective research, despite being labor intensive, costly, and prone to variability.

Objective

We developed and validated CIDER (Clinical Data Extractor), a secure, institutionally deployable, LLM-based pipeline for automated structured data extraction from routine clinical reports. We assessed the potential utility of the system for improving the completeness of clinical research datasets.

Methods

CIDER uses an asynchronous FastAPI-based architecture with a locally deployed vLLM inference engine running Qwen3-VL-32B-Instruct-FP8 model in an institution-controlled environment. The system was validated on 2073 real-world Hungarian-language histopathology reports (a challenging non-English setting), using a manually curated structured database as the reference standard. Seven variables were evaluated (sex, surgery year, T stage, N stage, organ, histology, and size). Extraction performance was assessed using exact-match accuracy, weighted F1-scores, Cohen κ statistics, and tolerance-based agreement thresholds for tumor size. Robustness was evaluated across temperatures from 0 to 2.0, and technical reproducibility was assessed at a temperature of 0.1 across 3 independent runs.

Results

The validation dataset comprised stand-alone native-text PDF pathology reports originating from multiple Hungarian oncology centers. Input document length showed a median of 3926 (mean 4258, SD 1057, IQR 3490-4688) tokens, while generated outputs contained a median of 63 (mean 62.5, SD 8.2, IQR 56-66) tokens. At a temperature of 0.1, CIDER achieved near-human agreement with expert-curated reference database, with exact-match accuracies of 99.5% for sex, 98.1% for surgery year, 95.8% for organ, 95.6% for T stage, 92.4% for N stage, 87.5% for histology, and 78.1% for tumor size. Weighted F1-scores ranged from 0.87 for histology to 0.995 for sex, while Cohen κ values ranged from 0.85 for N stage to 0.99 for sex. For tumor size extraction, 83.5% to 85.3% and 87.3% to 88.7% of predictions were within 5 mm and 10 mm of the manually curated values, respectively. CIDER additionally generated candidate extractions for variables omitted during manual curation, including 62.8% (713/1136) of missing T stages and 91.5% (289/316) of tumor size values. Sensitivity testing revealed high robustness, with negligible variance at temperature=0.1 and stable performance at high temperatures (temperature=2.0).

Conclusions

CIDER demonstrates that locally deployed open-weight LLMs can reliably extract structured clinical data from complex pathology reports while preserving institutional control over sensitive data. These findings support the feasibility of secure, institutionally deployable, LLM-based extraction systems for generating research-ready datasets, facilitating clinical registry development, improving dataset completeness, and enabling scalable reuse of unstructured clinical documentation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Data MiningInformation Storage and RetrievalLarge Language ModelsRetrospective StudiesHumansReproducibility of Results

Stichwörter der Autor:innen

LLMNLPbiobankingfree-text documentshealth care recordshistopathologylarge language modelnatural language processingoncologypatient documentation

Autor:innen und Institutionen

  • Institute of Molecular Life Sciences, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary.
  • Department of Bioinformatics, Semmelweis University, Budapest, Hungary.
  • Institute of Transdisciplinary Discoveries, Medical School, University of Pécs, Pécs, Hungary.

ORCID-Profile: Máté Posta, Aida Figler, Zsófia Dobolyi, Balázs Győrffy

Zitieren

APA

Posta, M., Figler, A., Dobolyi, Z., & Győrffy, B. (2026). A Secure, Scalable Large Language Model–Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study. Journal of Medical Internet Research. https://doi.org/10.2196/95780

Vancouver

Posta M, Figler A, Dobolyi Z, Győrffy B. A Secure, Scalable Large Language Model–Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study. Journal of Medical Internet Research. 2026. doi:10.2196/95780

BibTeX
@article{Posta2026,
  author = {Posta, Máté and Figler, Aida and Dobolyi, Zsófia and Győrffy, Balázs},
  title = {A Secure, Scalable Large Language Model–Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  doi = {10.2196/95780}
}
Kostenlos starten →