EUVIMEDEUVIMED entdecken
EvaluationsstudieOpen Access

Evaluating Large Language Models for Standardizing Clinical Data Using Health Level Seven Fast Healthcare Interoperability Resources: Development and Evaluation Study

Álvaro Riquelme-Tornel, Pedro Luis Costa Del Amo, Catalina Martínez-Costa

JMIR Medical Informatics · 13.11.2026 · 14 · e100368-e100368

DOI: 10.2196/100368 ↗

Kurz gefasst

Evaluationsstudie in der Zeitschrift JMIR Medical Informatics, online erschienen am 13.11.2026. Verfasst von Álvaro Riquelme-Tornel und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Informatics and Systems, Faculty of Computer Science, Universidad de Murcia, Campus de Espinardo, Murcia, 30100, Spain. Hauptthemen laut MEDLINE: Large Language Models, Health Information Interoperability, Health Level Seven. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

One of the barriers to the wider adoption of clinical information modeling standards is the high cost associated with the time and effort required to model and map data from legacy systems, as well as the need for specialized modeling experts. Meanwhile, large language models (LLMs) are becoming powerful tools for data transformation and semantic processing.

Objective

The aim of this study was to evaluate the capability of LLMs to support the mapping of structured clinical data to Health Level Seven (HL7) Fast Healthcare Interoperability Resources (FHIR) resources and attributes.

Methods

Using data from the MIMIC-IV (Medical Information Mart for Intensive Care IV) database, a data pipeline integrating clustering techniques, retrieval-augmented resource grounding, and schema-guided prompting was implemented to support the FHIR mapping of structured clinical data. The pipeline was evaluated under 2 scenarios with different levels of complexity. In the first scenario, used as a baseline, the input data were well described and contextually rich. In the second scenario, designed as a controlled test under substantially reduced context, the original table structure and table-level descriptions were removed, while data element names and LLM-enriched descriptions were retained.

Results

Using 65 data elements, top 2 resource identification accuracy reached 100% in the baseline scenario. The highest mean attribute-level mapping accuracy was 80.61% (SD 2.06) for GPT-4o and 69.54% (SD 5.15) for Llama 3.3 70B. In the low-context scenario, clustering-based context recovery increased top 5 resource identification accuracy to 94%. Across temperature settings, GPT-4o achieved mean attribute-level mapping accuracies ranging from 73.23% (SD 2.92) to 76.77% (SD 1.14), whereas Llama 3.3 70B ranged from 54.31% (SD 4.17) to 58.31% (SD 2.23). Error analysis showed that the main remaining failures were associated with resource-level confusion and terminal-attribute semantic ambiguity, whereas other error categories were comparatively infrequent.

Conclusions

LLMs can support structured clinical data mapping to HL7 FHIR, but fine-grained semantic disambiguation remains the principal limitation. In context-rich settings, the main errors occurred when selecting the appropriate terminal FHIR attribute. Under reduced context, accurate candidate-resource retrieval became increasingly important, and clustering-based context recovery substantially improved resource identification. These findings support the use of LLMs as assistive components within human-supervised interoperability workflows, with particular attention to candidate-resource selection and ambiguity-sensitive attribute mappings.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsHealth Information InteroperabilityHealth Level SevenHumansElectronic Health RecordsClustering Algorithms

Stichwörter der Autor:innen

Fast Healthcare Interoperability ResourcesHL7 FHIRHealth Level Sevenclinical data interoperabilityclusteringlarge language modelsretrieval-augmented generation

Autor:innen und Institutionen

  • Department of Informatics and Systems, Faculty of Computer Science, Universidad de Murcia, Campus de Espinardo, Murcia, 30100, Spain.
  • Instituto Murciano de Investigación Biosanitaria, Murcia, Spain.
  • Latency Data, Madrid, Spain.

Zitieren

APA

Riquelme-Tornel, Á., Costa Del Amo, P. L., & Martínez-Costa, C. (2026). Evaluating Large Language Models for Standardizing Clinical Data Using Health Level Seven Fast Healthcare Interoperability Resources: Development and Evaluation Study. JMIR Medical Informatics, 14, e100368-e100368. https://doi.org/10.2196/100368

Vancouver

Riquelme-Tornel Á, Costa Del Amo PL, Martínez-Costa C. Evaluating Large Language Models for Standardizing Clinical Data Using Health Level Seven Fast Healthcare Interoperability Resources: Development and Evaluation Study. JMIR Medical Informatics. 2026;14:e100368-e100368. doi:10.2196/100368

BibTeX
@article{RiquelmeTornel2026,
  author = {Riquelme-Tornel, Álvaro and Costa Del Amo, Pedro Luis and Martínez-Costa, Catalina},
  title = {Evaluating Large Language Models for Standardizing Clinical Data Using Health Level Seven Fast Healthcare Interoperability Resources: Development and Evaluation Study},
  journal = {JMIR Medical Informatics},
  year = {2026},
  volume = {14},
  pages = {e100368--e100368},
  doi = {10.2196/100368}
}
Kostenlos starten →