EUVIMEDEUVIMED entdecken
Open Access

Evaluating Large Language Models for Automated Data Cleaning and Feature Engineering in Clinical Datasets

Shimul A Babli, Pulkit Gairola, Chukwuemeka E Ogbu, Francesco Alessi Longa, Aditya Rautaray, Opeyemi S Alamu, Syed Faheem Haider Rizvi

Cureus · 09.05.2026

DOI: 10.7759/cureus.108550 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cureus, erschienen am 09.05.2026. Verfasst von Shimul A Babli und 6 weiteren Autor:innen. Beteiligte Einrichtung: Internal Medicine, Islami Bank Medical College and Hospital, Rajshahi, BGD. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: Electronic health records (EHRs) are increasingly used for clinical research and machine learning, yet they are plagued by missing values, outliers, inconsistent coding, and heterogeneous data types. Traditional rule-based cleaning pipelines demand extensive domain expertise and manual effort. Large language models (LLMs) exhibit strong code-generation and clinical reasoning abilities, but their utility for automating structured data preprocessing in clinical datasets remains underexplored. METHODS: We evaluated three LLMs, GPT-4 (OpenAI, Inc., San Francisco, United States), Claude 3.5 Sonnet (Anthropic, San Francisco, United States), and Gemini 1.5 Pro (Google DeepMind, Mountain View, United States), on five data cleaning tasks and two feature engineering tasks across three publicly available clinical datasets: Medical Information Mart for Intensive Care (MIMIC-IV) (PhysioNet), the eICU Collaborative Research Database (PhysioNet), and National Health and Nutrition Examination Survey (NHANES) 2017-2020 (CDC). LLM-generated preprocessing scripts were benchmarked against a conventional rule-based pipeline. Downstream predictive performance was assessed using XGBoost and logistic regression for in-hospital mortality prediction, evaluated by area under the receiver operating characteristic curve (AUROC) and F1 score. RESULTS: Claude 3.5 Sonnet achieved the highest mean data cleaning F1 score (0.90), followed by GPT-4 (0.89) and Gemini 1.5 Pro (0.85), all exceeding the rule-based baseline (0.77). For mortality prediction on MIMIC-IV, data preprocessed by Claude 3.5 Sonnet yielded the best XGBoost AUROC (0.851; 95% CI: 0.839-0.863), compared with GPT-4 (0.842), Gemini 1.5 Pro (0.829), rule-based cleaning (0.803), and no cleaning (0.761). LLM-engineered features contributed incremental AUROC gains of 0.015-0.025 over manually constructed feature sets. CONCLUSIONS: LLMs can substantially automate clinical data cleaning and feature engineering, achieving performance comparable to or exceeding hand-crafted pipelines. However, domain expert oversight remains essential to validate clinical plausibility and prevent silent data corruption.

Schlagwörter

Stichwörter der Autor:innen

clinical data cleaningelectronic health records (ehrs)feature engineeringlarge language models (llms)machine learning in healthcare

Autor:innen und Institutionen

  • Internal Medicine, Islami Bank Medical College and Hospital, Rajshahi, BGD.
  • Public Health, The University of Tennessee Knoxville, Knoxville, USA.
  • College of Medicine, The University of Tennessee Health Science Center, Knoxville, USA.
  • Internal Medicine, Cape Fear Valley Health, Fayetteville, USA.
  • International Law, Azteca University, Chalco, MEX.
  • Cloud Solutions Security, CVS Health, New York, USA.

Zitieren

APA

Babli, S. A., Gairola, P., Ogbu, C. E., Alessi Longa, F., Rautaray, A., Alamu, O. S., & Haider Rizvi, S. F. (2026). Evaluating Large Language Models for Automated Data Cleaning and Feature Engineering in Clinical Datasets. Cureus. https://doi.org/10.7759/cureus.108550

Vancouver

Babli SA, Gairola P, Ogbu CE, Alessi Longa F, Rautaray A, Alamu OS, et al. Evaluating Large Language Models for Automated Data Cleaning and Feature Engineering in Clinical Datasets. Cureus. 2026. doi:10.7759/cureus.108550

BibTeX
@article{Babli2026,
  author = {Babli, Shimul A and Gairola, Pulkit and Ogbu, Chukwuemeka E and Alessi Longa, Francesco and Rautaray, Aditya and Alamu, Opeyemi S and Haider Rizvi, Syed Faheem},
  title = {Evaluating Large Language Models for Automated Data Cleaning and Feature Engineering in Clinical Datasets},
  journal = {Cureus},
  year = {2026},
  doi = {10.7759/cureus.108550}
}
Kostenlos starten →