EUVIMEDEUVIMED entdecken
Open Access

Large Language Model–Assisted Semiautomated Data Harmonization Pipeline for the Multiple Chronic Disease Disparities Research Consortium: Framework Development and Comparative Validation Study

Hyelee Kim iD, Shuang Liang iD, Kathy Lanier iD, Sarit Helman iD, Nancy F Cheng iD, Niloufar Ameli, Jaysón Davidson iD, Shivani Mehta iD, Yulin Hswen iD, Catherine E Oldenburg iD, Kim F Rhoads iD, Edwin D Charlebois iD und 2 weitere

JMIR AI · 22.10.2026 · 5 · e93996-e93996

DOI: 10.2196/93996 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR AI, online erschienen am 22.10.2026. Verfasst von Hyelee Kim und 13 weiteren Autor:innen. Beteiligte Einrichtung: Department of Epidemiology and Biostatistics, University of California, 550 16th St., Floor 2, San Francisco, CA, 94143, United States. Die Arbeit nennt 48 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The increasing availability of machine-readable research data has created a growing need for efficient large-scale data harmonization (DH). Although large language models (LLMs) show promise for reducing the time and labor required for DH, their effective integration into harmonization workflows remains an important methodological challenge.

Objective

This study aims to develop and evaluate a semiautomated, human-in-the-loop (HITL) LLM-assisted DH workflow as a proof of concept across 8 projects within a research consortium focused on disparities in multiple chronic conditions.

Methods

We developed an LLM-assisted DH workflow that combined preprocessing, semantic mapping, variable response mapping, synthetic data generation, data transformation, and iterative researcher review. Using Generative Pretrained Transformer (GPT-4o) hosted in a secure academic environment, project-specific survey items were mapped to 114 Consortium Common Data Element (CDE) semantic groups. Semantic mapping accuracy was evaluated based on the Consortium’s consensus harmonization results. Complete and incomplete semantic mappings were distinguished according to construct overlap, and incomplete mappings requiring context-dependent human judgment were excluded for mapped item pairs and CDE semantic groups with and without HITL review.

Results

Following preprocessing, 884 survey items were included in the DH workflow, of which 812 were mapped to a mean of 79 CDE semantic groups per project. Mean semantic mapping accuracy for item pairs was 95.9% (SD 5.7%) with HITL review and 86.7% (SD 8.6%) without HITL review. At the CDE semantic-group level, the corresponding accuracies were 98.9% (SD 1.7%) and 94.9% (SD 4.0%), respectively. Across projects, omission of valid semantic mappings occurred more frequently than incorrect semantic mappings, particularly for heterogeneous response structures and subjective constructs measured using different instruments. Pipeline components—including concept-based preprocessing, iterative prompt refinement, and targeted HITL review—improved mapping completeness and supported variable harmonization across heterogeneous datasets.

Conclusions

This proof-of-concept study demonstrates that the effectiveness of LLM-assisted DH depends more on the design of a structured HITL workflow rather than on the LLM alone. Combining preprocessing, iterative verification, and targeted human oversight improved the efficiency and reliability of DH while preserving researcher judgment for complex mapping decisions.

Schlagwörter

Stichwörter der Autor:innen

social determinants of healthcommon data elementdata harmonizationhuman-in-the-looplarge language modelsemiautomation

Autor:innen und Institutionen

  • Department of Epidemiology and Biostatistics, University of California, 550 16th St., Floor 2, San Francisco, CA, 94143, United States.
  • Division of Preventive Science, Department of Medicine, University of California, San Francisco, CA, United States.
  • Research Coordinating Center for Reducing Disparities in Multiple Chronic Diseases, University of California, San Francisco, CA, United States.
  • Computational Precision Health, University of California San Francisco and University of California Berkeley, Berkeley, CA, United States.
  • Social and Behavioral Sciences Branch, Division of Intramural Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, BG 6710 Rockledge Dr Wing B, Bethesda, MD, 20817, United States, 1…
  • Division of Oral Epidemiology & Dental Public Health, University of California, San Francisco, CA, United States.

ORCID-Profile: Hyelee Kim, Shuang Liang, Kathy Lanier, Sarit Helman, Nancy F Cheng, Jaysón Davidson, Shivani Mehta, Yulin Hswen, Catherine E Oldenburg, Kim F Rhoads, Edwin D Charlebois, Stuart A Gansky, William Brown Iii

Zitieren

APA

Kim, H., Liang, S., Lanier, K., Helman, S., Cheng, N. F., Ameli, N., Davidson, J., Mehta, S., Hswen, Y., Oldenburg, C. E., Rhoads, K. F., Charlebois, E. D., Gansky, S. A., & Brown Iii, W. (2026). Large Language Model–Assisted Semiautomated Data Harmonization Pipeline for the Multiple Chronic Disease Disparities Research Consortium: Framework Development and Comparative Validation Study. JMIR AI, 5, e93996-e93996. https://doi.org/10.2196/93996

Vancouver

Kim H, Liang S, Lanier K, Helman S, Cheng NF, Ameli N, et al. Large Language Model–Assisted Semiautomated Data Harmonization Pipeline for the Multiple Chronic Disease Disparities Research Consortium: Framework Development and Comparative Validation Study. JMIR AI. 2026;5:e93996-e93996. doi:10.2196/93996

BibTeX
@article{Kim2026,
  author = {Kim, Hyelee and Liang, Shuang and Lanier, Kathy and Helman, Sarit and Cheng, Nancy F and Ameli, Niloufar and Davidson, Jaysón and Mehta, Shivani and Hswen, Yulin and Oldenburg, Catherine E and Rhoads, Kim F and Charlebois, Edwin D and Gansky, Stuart A and Brown Iii, William},
  title = {Large Language Model–Assisted Semiautomated Data Harmonization Pipeline for the Multiple Chronic Disease Disparities Research Consortium: Framework Development and Comparative Validation Study},
  journal = {JMIR AI},
  year = {2026},
  volume = {5},
  pages = {e93996--e93996},
  doi = {10.2196/93996}
}
Kostenlos starten →