EUVIMEDScopri EUVIMED
ValidierungsstudieOpen access

Using Structured Codes and Free-Text Notes to Measure Information Complementarity in Electronic Health Records: Feasibility and Validation Study

Tom M Seinen iD, Jan A Kors iD, Erik M van Mulligen iD, Peter R Rijnbeek iD

Journal of Medical Internet Research · 2025-02-13 · 27 · e66910

DOI: 10.2196/66910 ↗

At a glance

Validierungsstudie in Journal of Medical Internet Research, published online on 2025-02-13. Written by Tom M Seinen and 3 co-authors. Institution: Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, Netherlands. Major MEDLINE topics: Electronic Health Records. Cited 39 times so far (Crossref); 33 references. The full text is freely available in PubMed Central.

AbstractSubjectsDati bibliograficiCiteRelated studies

Abstract (original language)

Background

Electronic health records (EHRs) consist of both structured data (eg, diagnostic codes) and unstructured data (eg, clinical notes). It is commonly believed that unstructured clinical narratives provide more comprehensive information. However, this assumption lacks large-scale validation and direct validation methods.

Objective

This study aims to quantitatively compare the information in structured and unstructured EHR data and directly validate whether unstructured data offers more extensive information across a patient population.

Methods

We analyzed both structured and unstructured data from patient records and visits in a large Dutch primary care EHR database between January 2021 and January 2024. Clinical concepts were identified from free-text notes using an extraction framework tailored for Dutch and compared with concepts from structured data. Concept embeddings were generated to measure semantic similarity between structured and extracted concepts through cosine similarity. A similarity threshold was systematically determined via annotated matches and minimized weighted Gini impurity. We then quantified the concept overlap between structured and unstructured data across various concept domains and patient populations.

Results

In a population of 1.8 million patients, only 13% of extracted concepts from patient records and 7% from individual visits had similar structured counterparts. Conversely, 42% of structured concepts in records and 25% in visits had similar matches in unstructured data. Condition concepts had the highest overlap, followed by measurements and drug concepts. Subpopulation visits, such as those with chronic conditions or psychological disorders, showed different proportions of data overlap, indicating varied reliance on structured versus unstructured data across clinical contexts.

Conclusions

Our study demonstrates the feasibility of quantifying the information difference between structured and unstructured data, showing that the unstructured data provides important additional information in the studied database and populations. The annotated concept matches are made publicly available for the clinical natural language processing community. Despite some limitations, our proposed methodology proves versatile, and its application can lead to more robust and insightful observational clinical research.

Subjects

MeSH terms (MEDLINE)

Electronic Health RecordsHumansFeasibility StudiesNetherlands

Author keywords

EHRclinical concept extractionclinical concept similaritycodedataelectronic health recordselectronic recordframeworkfree-textinformationmachine learningnamed entity recognitionnatural language processingpatient recordsstructured data

Authors and institutions

  • Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, Netherlands.

ORCID profiles: Tom M Seinen, Jan A Kors, Erik M van Mulligen, Peter R Rijnbeek

Cite

APA

Seinen, T. M., Kors, J. A., van Mulligen, E. M., & Rijnbeek, P. R. (2025). Using Structured Codes and Free-Text Notes to Measure Information Complementarity in Electronic Health Records: Feasibility and Validation Study. Journal of Medical Internet Research, 27, e66910. https://doi.org/10.2196/66910

Vancouver

Seinen TM, Kors JA, van Mulligen EM, Rijnbeek PR. Using Structured Codes and Free-Text Notes to Measure Information Complementarity in Electronic Health Records: Feasibility and Validation Study. Journal of Medical Internet Research. 2025;27:e66910. doi:10.2196/66910

BibTeX
@article{Seinen2025,
  author = {Seinen, Tom M and Kors, Jan A and van Mulligen, Erik M and Rijnbeek, Peter R},
  title = {Using Structured Codes and Free-Text Notes to Measure Information Complementarity in Electronic Health Records: Feasibility and Validation Study},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e66910},
  doi = {10.2196/66910}
}
Start for free →