EUVIMEDScopri EUVIMED
Open access

AI-Assisted Clinical Data Abstraction From Electronic Health Records: Retrospective Concordance Study

Camille Sarah Schwartz iD, Michael John Anderson iD, Kelsey Nicole Moakler iD, Bradley Adam Newby iD, David Alan Davenport iD, Matthew Wilson Schwartz iD

JMIR Formative Research · 2026-07-07 · 10 · e96755-e96755

DOI: 10.2196/96755 ↗

At a glance

Journal article in JMIR Formative Research, published online on 2026-07-07. Written by Camille Sarah Schwartz and 5 co-authors. Institution: University of Nevada, Reno, 1664 N Virginia St, Reno, NV, 89557, United States, 1 7027529240. Major MEDLINE topics: Electronic Health Records, Artificial Intelligence. 25 references. The full text is freely available in PubMed Central.

AbstractSubjectsDati bibliograficiCiteRelated studies

Abstract (original language)

Background

Manual chart abstraction from electronic health records is a critical step in clinical outcomes research but is time-intensive and prone to human error. Advances in artificial intelligence (AI), particularly large language models, offer the potential to automate the extraction of structured data from unstructured clinical documentation with improved efficiency and consistency.

Objective

This study aimed to evaluate the accuracy and efficiency of an AI-assisted approach for extracting patient-reported outcomes from clinical notes compared with traditional human abstraction.

Methods

We conducted a retrospective study of 26 patients treated with low-dose radiation therapy for osteoarthritis. Human reviewers abstracted numeric rating scale (NRS; 0‐10) pain scores at baseline, the end of treatment, and the first follow-up, and von Pannewitz score (VPS; 0‐4) improvement scores at posttreatment time points. A HIPAA (Health Insurance Portability and Accountability Act)–compliant generative pretrained transformer–based AI system was prompted to extract the same end points from clinical notes. Concordance was assessed using exact match rates, the intraclass correlation coefficient for the NRS, and weighted Cohen κ for the VPS. The time required for AI vs manual abstraction was recorded. The AI system was not trained or fine-tuned on study data, and performance was evaluated directly against human abstraction to reflect real-world deployment.

Results

The AI system demonstrated high concordance with human abstraction, achieving an exact match rate of 92% for the NRS (95% CI 84‐96; intraclass correlation coefficient=0.96) and 94% for the VPS (95% CI 84‐98; κ=0.91). All discrepancies were minor, and no spurious values were generated. The AI system identified 1 clinically relevant data point missed during manual review. Average abstraction time per patient decreased from approximately 30 minutes to 2 minutes, representing time savings of >90%. The system also captured trends in analgesic use, but these results were not statistically significant, including reductions without escalation.

Conclusions

AI-assisted data abstraction demonstrated high concordance with human review in this single-institution cohort while substantially reducing the time requirements. These findings support the feasibility of AI-assisted abstraction workflows, although further validation across larger and more diverse datasets is needed.

Subjects

MeSH terms (MEDLINE)

Electronic Health RecordsArtificial IntelligenceHumansRetrospective StudiesFemaleMaleAgedMiddle AgedIntelligent SystemsGenerative Artificial IntelligenceOsteoarthritis

Author keywords

AIartificial intelligenceclinical data abstractionelectronic health recordslarge language modelsmedical informaticsnatural language processing

Authors and institutions

  • University of Nevada, Reno, 1664 N Virginia St, Reno, NV, 89557, United States, 1 7027529240.
  • Comprehensive Cancer Centers of Nevada (CCCN), Henderson, NV, United States.
  • Kirk Kerkorian School of Medicine, University of Nevada, Las Vegas, Las Vegas, NV, United States.

ORCID profiles: Camille Sarah Schwartz, Michael John Anderson, Kelsey Nicole Moakler, Bradley Adam Newby, David Alan Davenport, Matthew Wilson Schwartz

Cite

APA

Schwartz, C. S., Anderson, M. J., Moakler, K. N., Newby, B. A., Davenport, D. A., & Schwartz, M. W. (2026). AI-Assisted Clinical Data Abstraction From Electronic Health Records: Retrospective Concordance Study. JMIR Formative Research, 10, e96755-e96755. https://doi.org/10.2196/96755

Vancouver

Schwartz CS, Anderson MJ, Moakler KN, Newby BA, Davenport DA, Schwartz MW. AI-Assisted Clinical Data Abstraction From Electronic Health Records: Retrospective Concordance Study. JMIR Formative Research. 2026;10:e96755-e96755. doi:10.2196/96755

BibTeX
@article{Schwartz2026,
  author = {Schwartz, Camille Sarah and Anderson, Michael John and Moakler, Kelsey Nicole and Newby, Bradley Adam and Davenport, David Alan and Schwartz, Matthew Wilson},
  title = {AI-Assisted Clinical Data Abstraction From Electronic Health Records: Retrospective Concordance Study},
  journal = {JMIR Formative Research},
  year = {2026},
  volume = {10},
  pages = {e96755--e96755},
  doi = {10.2196/96755}
}
Start for free →