EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Comparison of Large Language Models with Rules-Based Natural Language Processing Algorithms for Extracting Data from Operative Notes

Linjun Yang iD, Kellen L Mulford iD, Miguel M Girod-Hoffman iD, Monty Khela iD, Amirali Khosravi iD, Derek M Crossman iD, Anish Kanabar iD, Sami Saniei iD, Marisa N Ulrich, Michael J Taunton iD, Cody C Wyles iD

Journal of Bone and Joint Surgery · 2026

DOI: 10.2106/jbjs.25.01338 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Journal of Bone and Joint Surgery, 2026. Verfasst von Linjun Yang und 10 weiteren Autor:innen. Beteiligte Einrichtung: Orthopedic Surgery Artificial Intelligence Laboratory, Department of Orthopedic Surgery, Mayo Clinic, Rochester, Minnesota. Hauptthemen laut MEDLINE: Natural Language Processing, Large Language Models, Arthroplasty, Replacement, Hip, Algorithms. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 11 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIAMS NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

We aimed to develop automated data extraction pipelines with large language models (LLMs) to extract registry data from total hip arthroplasty (THA) operative notes and compare the performance with that of existing natural language processing (NLP) algorithms.

Methods

We randomly sampled 1,000 primary THA cases from our institutional registry. Two human annotators manually reviewed each operative note for 3 data points: surgical approach, bearing surface, and fixation technique. All labeled THA notes were split into the development set (n = 239) and the testing set (n = 719). We developed a custom data extraction pipeline for each data point by combining an iteratively customized prompt with an LLM. The performance was compared with that of existing rules-based NLP algorithms.

Results

The accuracy of LLMs was superior to that of NLP algorithms for all data points: surgical approach (96% compared with 94%), bearing surface (89% compared with 74%), and fixation technique (96% compared with 95%). Furthermore, the LLM accurately inferred the bearing surface for 80% of the notes that were ambiguous about the bearing surface.

Conclusions

We developed LLM pipelines for extracting 3 registry-relevant data points from THA operative notes, demonstrating superior performance to existing NLP algorithms. Clinical Relevance: LLMs have the potential to impact clinical care, including the evaluation of electronic medical record free-text data. As registries serve as a cornerstone of orthopaedic evidence, this work demonstrates promise for LLMs to simplify, improve, and democratize the construction of registry databases from operative notes.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingLarge Language ModelsArthroplasty, Replacement, HipAlgorithmsHumansRegistriesElectronic Health Records

Autor:innen und Institutionen

  • Orthopedic Surgery Artificial Intelligence Laboratory, Department of Orthopedic Surgery, Mayo Clinic, Rochester, Minnesota.
  • Department of Orthopedic Surgery, Mayo Clinic, Rochester, Minnesota.

ORCID-Profile: Linjun Yang, Kellen L Mulford, Miguel M Girod-Hoffman, Monty Khela, Amirali Khosravi, Derek M Crossman, Anish Kanabar, Sami Saniei, Michael J Taunton, Cody C Wyles

Förderung

  • NIAMS NIH HHS

Zitieren

APA

Yang, L., Mulford, K. L., Girod-Hoffman, M. M., Khela, M., Khosravi, A., Crossman, D. M., Kanabar, A., Saniei, S., Ulrich, M. N., Taunton, M. J., & Wyles, C. C. (2026). Comparison of Large Language Models with Rules-Based Natural Language Processing Algorithms for Extracting Data from Operative Notes. Journal of Bone and Joint Surgery. https://doi.org/10.2106/jbjs.25.01338

Vancouver

Yang L, Mulford KL, Girod-Hoffman MM, Khela M, Khosravi A, Crossman DM, et al. Comparison of Large Language Models with Rules-Based Natural Language Processing Algorithms for Extracting Data from Operative Notes. Journal of Bone and Joint Surgery. 2026. doi:10.2106/jbjs.25.01338

BibTeX
@article{Yang2026,
  author = {Yang, Linjun and Mulford, Kellen L and Girod-Hoffman, Miguel M and Khela, Monty and Khosravi, Amirali and Crossman, Derek M and Kanabar, Anish and Saniei, Sami and Ulrich, Marisa N and Taunton, Michael J and Wyles, Cody C},
  title = {Comparison of Large Language Models with Rules-Based Natural Language Processing Algorithms for Extracting Data from Operative Notes},
  journal = {Journal of Bone and Joint Surgery},
  year = {2026},
  doi = {10.2106/jbjs.25.01338}
}
Kostenlos starten →