Evaluating Large Language Models for Automated Data Cleaning and Feature Engineering in Clinical Datasets
Cureus · 09.05.2026
DOI: 10.7759/cureus.108550 ↗Kurz gefasst
Fachartikel in der Zeitschrift Cureus, erschienen am 09.05.2026. Verfasst von Shimul A Babli und 6 weiteren Autor:innen. Beteiligte Einrichtung: Internal Medicine, Islami Bank Medical College and Hospital, Rajshahi, BGD. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
BACKGROUND: Electronic health records (EHRs) are increasingly used for clinical research and machine learning, yet they are plagued by missing values, outliers, inconsistent coding, and heterogeneous data types. Traditional rule-based cleaning pipelines demand extensive domain expertise and manual effort. Large language models (LLMs) exhibit strong code-generation and clinical reasoning abilities, but their utility for automating structured data preprocessing in clinical datasets remains underexplored. METHODS: We evaluated three LLMs, GPT-4 (OpenAI, Inc., San Francisco, United States), Claude 3.5 Sonnet (Anthropic, San Francisco, United States), and Gemini 1.5 Pro (Google DeepMind, Mountain View, United States), on five data cleaning tasks and two feature engineering tasks across three publicly available clinical datasets: Medical Information Mart for Intensive Care (MIMIC-IV) (PhysioNet), the eICU Collaborative Research Database (PhysioNet), and National Health and Nutrition Examination Survey (NHANES) 2017-2020 (CDC). LLM-generated preprocessing scripts were benchmarked against a conventional rule-based pipeline. Downstream predictive performance was assessed using XGBoost and logistic regression for in-hospital mortality prediction, evaluated by area under the receiver operating characteristic curve (AUROC) and F1 score. RESULTS: Claude 3.5 Sonnet achieved the highest mean data cleaning F1 score (0.90), followed by GPT-4 (0.89) and Gemini 1.5 Pro (0.85), all exceeding the rule-based baseline (0.77). For mortality prediction on MIMIC-IV, data preprocessed by Claude 3.5 Sonnet yielded the best XGBoost AUROC (0.851; 95% CI: 0.839-0.863), compared with GPT-4 (0.842), Gemini 1.5 Pro (0.829), rule-based cleaning (0.803), and no cleaning (0.761). LLM-engineered features contributed incremental AUROC gains of 0.015-0.025 over manually constructed feature sets. CONCLUSIONS: LLMs can substantially automate clinical data cleaning and feature engineering, achieving performance comparable to or exceeding hand-crafted pipelines. However, domain expert oversight remains essential to validate clinical plausibility and prevent silent data corruption.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Internal Medicine, Islami Bank Medical College and Hospital, Rajshahi, BGD.
- Public Health, The University of Tennessee Knoxville, Knoxville, USA.
- College of Medicine, The University of Tennessee Health Science Center, Knoxville, USA.
- Internal Medicine, Cape Fear Valley Health, Fayetteville, USA.
- International Law, Azteca University, Chalco, MEX.
- Cloud Solutions Security, CVS Health, New York, USA.
Zitieren
APA
Babli, S. A., Gairola, P., Ogbu, C. E., Alessi Longa, F., Rautaray, A., Alamu, O. S., & Haider Rizvi, S. F. (2026). Evaluating Large Language Models for Automated Data Cleaning and Feature Engineering in Clinical Datasets. Cureus. https://doi.org/10.7759/cureus.108550
Vancouver
Babli SA, Gairola P, Ogbu CE, Alessi Longa F, Rautaray A, Alamu OS, et al. Evaluating Large Language Models for Automated Data Cleaning and Feature Engineering in Clinical Datasets. Cureus. 2026. doi:10.7759/cureus.108550
BibTeX
@article{Babli2026,
author = {Babli, Shimul A and Gairola, Pulkit and Ogbu, Chukwuemeka E and Alessi Longa, Francesco and Rautaray, Aditya and Alamu, Opeyemi S and Haider Rizvi, Syed Faheem},
title = {Evaluating Large Language Models for Automated Data Cleaning and Feature Engineering in Clinical Datasets},
journal = {Cureus},
year = {2026},
doi = {10.7759/cureus.108550}
}
EUVIMED