EUVIMEDEUVIMED entdecken
Open Access

Ensuring Reliability of Curated Electronic Health Record–Derived Data: The Validation of Accuracy for Large Language Model–/Machine Learning–Extracted Information and Data (VALID) Framework

Melissa Estevez iD, Nisha Singh, Lauren Dyson, Blythe Adamson iD, Konstantin Krismer iD, Kelly Magee, Qianyu Yuan, Megan W Hildner, Erin Fidyk iD, Tori Williams, Olive Mbah iD, Farhad Khan und 2 weitere

JCO Clinical Cancer Informatics · 2026-01 · 10 (1)

DOI: 10.1200/cci-25-00215 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JCO Clinical Cancer Informatics, 2026-01. Verfasst von Melissa Estevez und 13 weiteren Autor:innen. Beteiligte Einrichtung: Flatiron Health, New York, NY. Hauptthemen laut MEDLINE: Electronic Health Records, Machine Learning, Data Curation. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) are increasingly used to extract clinical data from electronic health records, offering significant improvements in scalability and efficiency for real-world data (RWD) curation in oncology. However, the adoption of LLMs introduces new challenges in ensuring the reliability, accuracy, and fairness of extracted data, which are essential for research, regulatory, and clinical applications. Existing quality assurance frameworks for RWD and artificial intelligence (AI) do not fully address the unique error modes and complexities associated with LLM-extracted data. In this paper, we propose a comprehensive framework for evaluating the quality of clinical data extracted by LLMs. The framework integrates variable-level performance benchmarking against expert human abstraction, verification checks for internal consistency and plausibility, and replication analyses comparing LLM-extracted data to human-abstracted data sets or external standards. This multidimensional approach enables the identification of variables most in need of improvement, systematic detection of latent errors, and confirmation of data set fitness-for-purpose in real-world research. Additionally, the framework supports bias assessment by stratifying across demographic subgroups. By providing a rigorous and transparent method for assessing LLM-extracted RWD, this framework advances industry standards and supports the trustworthy use of AI-powered evidence generation in oncology research and practice.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Electronic Health RecordsMachine LearningData CurationLarge Language ModelsHumansReproducibility of ResultsArtificial Intelligence

Autor:innen und Institutionen

  • Flatiron Health, New York, NY.
  • Flatiron Health GmbH, Berlin, Germany.

ORCID-Profile: Melissa Estevez, Blythe Adamson, Konstantin Krismer, Erin Fidyk, Olive Mbah, Aaron B Cohen

Zitieren

APA

Estevez, M., Singh, N., Dyson, L., Adamson, B., Krismer, K., Magee, K., Yuan, Q., Hildner, M. W., Fidyk, E., Williams, T., Mbah, O., Khan, F., Seidl-Rathkopf, K., & Cohen, A. B. (2026). Ensuring Reliability of Curated Electronic Health Record–Derived Data: The Validation of Accuracy for Large Language Model–/Machine Learning–Extracted Information and Data (VALID) Framework. JCO Clinical Cancer Informatics, 10(1). https://doi.org/10.1200/cci-25-00215

Vancouver

Estevez M, Singh N, Dyson L, Adamson B, Krismer K, Magee K, et al. Ensuring Reliability of Curated Electronic Health Record–Derived Data: The Validation of Accuracy for Large Language Model–/Machine Learning–Extracted Information and Data (VALID) Framework. JCO Clinical Cancer Informatics. 2026;10(1). doi:10.1200/cci-25-00215

BibTeX
@article{Estevez2026,
  author = {Estevez, Melissa and Singh, Nisha and Dyson, Lauren and Adamson, Blythe and Krismer, Konstantin and Magee, Kelly and Yuan, Qianyu and Hildner, Megan W and Fidyk, Erin and Williams, Tori and Mbah, Olive and Khan, Farhad and Seidl-Rathkopf, Kathi and Cohen, Aaron B},
  title = {Ensuring Reliability of Curated Electronic Health Record–Derived Data: The Validation of Accuracy for Large Language Model–/Machine Learning–Extracted Information and Data (VALID) Framework},
  journal = {JCO Clinical Cancer Informatics},
  year = {2026},
  volume = {10},
  number = {1},
  doi = {10.1200/cci-25-00215}
}
Kostenlos starten →