Evaluating Large Language Models for Standardizing Clinical Data Using Health Level Seven Fast Healthcare Interoperability Resources: Development and Evaluation Study
JMIR Medical Informatics · 13.11.2026 · 14 · e100368-e100368
DOI: 10.2196/100368 ↗Kurz gefasst
Evaluationsstudie in der Zeitschrift JMIR Medical Informatics, online erschienen am 13.11.2026. Verfasst von Álvaro Riquelme-Tornel und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Informatics and Systems, Faculty of Computer Science, Universidad de Murcia, Campus de Espinardo, Murcia, 30100, Spain. Hauptthemen laut MEDLINE: Large Language Models, Health Information Interoperability, Health Level Seven. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
One of the barriers to the wider adoption of clinical information modeling standards is the high cost associated with the time and effort required to model and map data from legacy systems, as well as the need for specialized modeling experts. Meanwhile, large language models (LLMs) are becoming powerful tools for data transformation and semantic processing.
Objective
The aim of this study was to evaluate the capability of LLMs to support the mapping of structured clinical data to Health Level Seven (HL7) Fast Healthcare Interoperability Resources (FHIR) resources and attributes.
Methods
Using data from the MIMIC-IV (Medical Information Mart for Intensive Care IV) database, a data pipeline integrating clustering techniques, retrieval-augmented resource grounding, and schema-guided prompting was implemented to support the FHIR mapping of structured clinical data. The pipeline was evaluated under 2 scenarios with different levels of complexity. In the first scenario, used as a baseline, the input data were well described and contextually rich. In the second scenario, designed as a controlled test under substantially reduced context, the original table structure and table-level descriptions were removed, while data element names and LLM-enriched descriptions were retained.
Results
Using 65 data elements, top 2 resource identification accuracy reached 100% in the baseline scenario. The highest mean attribute-level mapping accuracy was 80.61% (SD 2.06) for GPT-4o and 69.54% (SD 5.15) for Llama 3.3 70B. In the low-context scenario, clustering-based context recovery increased top 5 resource identification accuracy to 94%. Across temperature settings, GPT-4o achieved mean attribute-level mapping accuracies ranging from 73.23% (SD 2.92) to 76.77% (SD 1.14), whereas Llama 3.3 70B ranged from 54.31% (SD 4.17) to 58.31% (SD 2.23). Error analysis showed that the main remaining failures were associated with resource-level confusion and terminal-attribute semantic ambiguity, whereas other error categories were comparatively infrequent.
Conclusions
LLMs can support structured clinical data mapping to HL7 FHIR, but fine-grained semantic disambiguation remains the principal limitation. In context-rich settings, the main errors occurred when selecting the appropriate terminal FHIR attribute. Under reduced context, accurate candidate-resource retrieval became increasingly important, and clustering-based context recovery substantially improved resource identification. These findings support the use of LLMs as assistive components within human-supervised interoperability workflows, with particular attention to candidate-resource selection and ambiguity-sensitive attribute mappings.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Informatics and Systems, Faculty of Computer Science, Universidad de Murcia, Campus de Espinardo, Murcia, 30100, Spain.
- Instituto Murciano de Investigación Biosanitaria, Murcia, Spain.
- Latency Data, Madrid, Spain.
Zitieren
APA
Riquelme-Tornel, Á., Costa Del Amo, P. L., & Martínez-Costa, C. (2026). Evaluating Large Language Models for Standardizing Clinical Data Using Health Level Seven Fast Healthcare Interoperability Resources: Development and Evaluation Study. JMIR Medical Informatics, 14, e100368-e100368. https://doi.org/10.2196/100368
Vancouver
Riquelme-Tornel Á, Costa Del Amo PL, Martínez-Costa C. Evaluating Large Language Models for Standardizing Clinical Data Using Health Level Seven Fast Healthcare Interoperability Resources: Development and Evaluation Study. JMIR Medical Informatics. 2026;14:e100368-e100368. doi:10.2196/100368
BibTeX
@article{RiquelmeTornel2026,
author = {Riquelme-Tornel, Álvaro and Costa Del Amo, Pedro Luis and Martínez-Costa, Catalina},
title = {Evaluating Large Language Models for Standardizing Clinical Data Using Health Level Seven Fast Healthcare Interoperability Resources: Development and Evaluation Study},
journal = {JMIR Medical Informatics},
year = {2026},
volume = {14},
pages = {e100368--e100368},
doi = {10.2196/100368}
}
EUVIMED