EUVIMEDEUVIMED entdecken
EvaluationsstudieOpen Access

Manual Evaluation of Record Linkage Algorithm Performance in Four Real-World Datasets

Agrayan K Gupta, Huiping Xu, Xiaochun Li, Joshua R Vest, Shaun J Grannis

Applied Clinical Informatics · 01.05.2024 · 15 (03) · 620-628

DOI: 10.1055/a-2291-1391 ↗

Kurz gefasst

Evaluationsstudie in der Zeitschrift Applied Clinical Informatics, erschienen am 01.05.2024. Verfasst von Agrayan K Gupta und 4 weiteren Autor:innen. Beteiligte Einrichtung: Indiana University School of Medicine, Indianapolis, Indiana, United States. Hauptthemen laut MEDLINE: Algorithms. Die Arbeit nennt 28 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Patient-Centered Outcomes Research Institute.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

Patient data are fragmented across multiple repositories, yielding suboptimal and costly care. Record linkage algorithms are widely accepted solutions for improving completeness of patient records. However, studies often fail to fully describe their linkage techniques. Further, while many frameworks evaluate record linkage methods, few focus on producing gold standard datasets. This highlights a need to assess these frameworks and their real-world performance. We use real-world datasets and expand upon previous frameworks to evaluate a consistent approach to the manual review of gold standard datasets and measure its impact on algorithm performance.

Methods

We applied the framework, which includes elements for data description, reviewer training and adjudication, and software and reviewer descriptions, to four datasets. Record pairs were formed and between 15,000 and 16,500 records were randomly sampled from these pairs. After training, two reviewers determined match status for each record pair. If reviewers disagreed, a third reviewer was used for final adjudication.

Results

Between the four datasets, the percent discordant rate ranged from 1.8 to 13.6%. While reviewers' discordance rate typically ranged between 1 and 5%, one exhibited a 59% discordance rate, showing the importance of the third reviewer. The original analysis was compared with three sensitivity analyses. The original analysis most often exhibited the highest predictive values compared with the sensitivity analyses.

Conclusion

Reviewers vary in their assessment of a gold standard, which can lead to variances in estimates for matching performance. Our analysis demonstrates how a multireviewer process can be applied to create gold standards, identify reviewer discrepancies, and evaluate algorithm performance.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AlgorithmsHumansMedical Record LinkageDatasets as TopicElectronic Health Records

Autor:innen und Institutionen

  • Indiana University School of Medicine, Indianapolis, Indiana, United States.
  • Department of Biostatistics, Indiana University School of Medicine, Indianapolis, Indiana, United States.
  • Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, Indiana, United States.
  • Department of Health Policy and Management, Indiana University Richard M. Fairbanks School of Public Health, Indianapolis, Indiana, United States.

Förderung

  • Patient-Centered Outcomes Research Institute

Zitieren

APA

Gupta, A. K., Xu, H., Li, X., Vest, J. R., & Grannis, S. J. (2024). Manual Evaluation of Record Linkage Algorithm Performance in Four Real-World Datasets. Applied Clinical Informatics, 15(03), 620-628. https://doi.org/10.1055/a-2291-1391

Vancouver

Gupta AK, Xu H, Li X, Vest JR, Grannis SJ. Manual Evaluation of Record Linkage Algorithm Performance in Four Real-World Datasets. Applied Clinical Informatics. 2024;15(03):620-628. doi:10.1055/a-2291-1391

BibTeX
@article{Gupta2024,
  author = {Gupta, Agrayan K and Xu, Huiping and Li, Xiaochun and Vest, Joshua R and Grannis, Shaun J},
  title = {Manual Evaluation of Record Linkage Algorithm Performance in Four Real-World Datasets},
  journal = {Applied Clinical Informatics},
  year = {2024},
  volume = {15},
  number = {03},
  pages = {620--628},
  doi = {10.1055/a-2291-1391}
}
Kostenlos starten →