Automating the segmentation, date extraction, and classification of multi-report PDFs in outside medical records using optical character recognition and generative artificial intelligence
JAMIA Open · 06.03.2026 · 9 (2)
DOI: 10.1093/jamiaopen/ooag027 ↗Kurz gefasst
Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 31.03.2026. Verfasst von Shivam Damani und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medicine, Mayo Clinic, Rochester, MN, 55905, United States. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Objectives
Patients referred for specialized care often arrive with outside medical records (OMRs) compiled into multi-report PDFs that include imaging, pathology, and clinical notes in unstructured formats. Reviewing these records is time consuming and mentally taxing, increasing the risk of delayed care, clinician frustration, and missed information affecting quality of care. This study aimed to automate the segmentation, classification, and date extraction of scanned OMRs, with a focus on records relevant to breast cancer care. Materials and Methods We used optical character recognition (OCR) to extract machine-readable text from 1303 scanned PDF documents from 116 distinct external institutions. Gemini 1.5, a large language model (LLM), was then used to segment multi-report files into individual documents, classify them into clinically meaningful categories such as mammograms and pathology reports, and extract study dates to build diagnostic timelines. Document categories were informed by clinical workflows in a breast cancer center.
Results
The system achieved an F1 score of 0.95 for segmentation, 0.96 for classification, and 0.90 for date extraction. In a pilot of 45 records reviewed by clinicians, only 2 classification errors and 1 date error were reported. Clinicians estimated that the tool reduced OMR review time by 40%, improved workflow efficiency, and increased satisfaction.
Discussion
Our findings demonstrate that combining OCR with LLMs can significantly enhance the processing of unstructured medical records, reducing manual burden and supporting timely clinical decision-making.
Conclusion
This study demonstrates the successful application of OCR and LLMs for organizing scanned OMRs within a specialty clinic. By automating a previously manual process, the approach supports scalable review of incoming outside records and has potential for adaptation to other clinical workflows. Future work will focus on evaluating the system across additional specialties and institutions.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Medicine, Mayo Clinic, Rochester, MN, 55905, United States.
- Center for Digital Health, Mayo Clinic, Rochester, MN, 55905, United States.
- Department of Surgery, Mayo Clinic, Jacksonville, FL, 55905, United States.
ORCID-Profile: Alexander Ryu
Zitieren
APA
Damani, S., Hinton, B., Hunt, T., Lawrence, N., Miller, K., Rice, M., Peterson, K., McLaughlin, S., & Ryu, A. (2026). Automating the segmentation, date extraction, and classification of multi-report PDFs in outside medical records using optical character recognition and generative artificial intelligence. JAMIA Open, 9(2). https://doi.org/10.1093/jamiaopen/ooag027
Vancouver
Damani S, Hinton B, Hunt T, Lawrence N, Miller K, Rice M, et al. Automating the segmentation, date extraction, and classification of multi-report PDFs in outside medical records using optical character recognition and generative artificial intelligence. JAMIA Open. 2026;9(2). doi:10.1093/jamiaopen/ooag027
BibTeX
@article{Damani2026,
author = {Damani, Shivam and Hinton, Benjamin and Hunt, Tanner and Lawrence, Nicholas and Miller, Kurt and Rice, Melinda and Peterson, Kevin and McLaughlin, Sarah and Ryu, Alexander},
title = {Automating the segmentation, date extraction, and classification of multi-report PDFs in outside medical records using optical character recognition and generative artificial intelligence},
journal = {JAMIA Open},
year = {2026},
volume = {9},
number = {2},
doi = {10.1093/jamiaopen/ooag027}
}
EUVIMED