Using Structured Codes and Free-Text Notes to Measure Information Complementarity in Electronic Health Records: Feasibility and Validation Study
Journal of Medical Internet Research · 2025-02-13 · 27 · e66910
DOI: 10.2196/66910 ↗At a glance
Validierungsstudie in Journal of Medical Internet Research, published online on 2025-02-13. Written by Tom M Seinen and 3 co-authors. Institution: Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, Netherlands. Major MEDLINE topics: Electronic Health Records. Cited 39 times so far (Crossref); 33 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Electronic health records (EHRs) consist of both structured data (eg, diagnostic codes) and unstructured data (eg, clinical notes). It is commonly believed that unstructured clinical narratives provide more comprehensive information. However, this assumption lacks large-scale validation and direct validation methods.
Objective
This study aims to quantitatively compare the information in structured and unstructured EHR data and directly validate whether unstructured data offers more extensive information across a patient population.
Methods
We analyzed both structured and unstructured data from patient records and visits in a large Dutch primary care EHR database between January 2021 and January 2024. Clinical concepts were identified from free-text notes using an extraction framework tailored for Dutch and compared with concepts from structured data. Concept embeddings were generated to measure semantic similarity between structured and extracted concepts through cosine similarity. A similarity threshold was systematically determined via annotated matches and minimized weighted Gini impurity. We then quantified the concept overlap between structured and unstructured data across various concept domains and patient populations.
Results
In a population of 1.8 million patients, only 13% of extracted concepts from patient records and 7% from individual visits had similar structured counterparts. Conversely, 42% of structured concepts in records and 25% in visits had similar matches in unstructured data. Condition concepts had the highest overlap, followed by measurements and drug concepts. Subpopulation visits, such as those with chronic conditions or psychological disorders, showed different proportions of data overlap, indicating varied reliance on structured versus unstructured data across clinical contexts.
Conclusions
Our study demonstrates the feasibility of quantifying the information difference between structured and unstructured data, showing that the unstructured data provides important additional information in the studied database and populations. The annotated concept matches are made publicly available for the clinical natural language processing community. Despite some limitations, our proposed methodology proves versatile, and its application can lead to more robust and insightful observational clinical research.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, Netherlands.
ORCID profiles: Tom M Seinen, Jan A Kors, Erik M van Mulligen, Peter R Rijnbeek
Cite
APA
Seinen, T. M., Kors, J. A., van Mulligen, E. M., & Rijnbeek, P. R. (2025). Using Structured Codes and Free-Text Notes to Measure Information Complementarity in Electronic Health Records: Feasibility and Validation Study. Journal of Medical Internet Research, 27, e66910. https://doi.org/10.2196/66910
Vancouver
Seinen TM, Kors JA, van Mulligen EM, Rijnbeek PR. Using Structured Codes and Free-Text Notes to Measure Information Complementarity in Electronic Health Records: Feasibility and Validation Study. Journal of Medical Internet Research. 2025;27:e66910. doi:10.2196/66910
BibTeX
@article{Seinen2025,
author = {Seinen, Tom M and Kors, Jan A and van Mulligen, Erik M and Rijnbeek, Peter R},
title = {Using Structured Codes and Free-Text Notes to Measure Information Complementarity in Electronic Health Records: Feasibility and Validation Study},
journal = {Journal of Medical Internet Research},
year = {2025},
volume = {27},
pages = {e66910},
doi = {10.2196/66910}
}
EUVIMED