Automated identification of radiotherapy treatment sites from unstructured physician notes
Journal of Applied Clinical Medical Physics · 2026-04 · 27 (4)
DOI: 10.1002/acm2.70558 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Applied Clinical Medical Physics, online erschienen am 31.03.2026. Verfasst von Klea Hoxha und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, University of California, Los Angeles, Los Angeles, California, USA. Hauptthemen laut MEDLINE: Radiotherapy Planning, Computer-Assisted, Neoplasms, Documentation, Radiation Oncology. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Purpose
Ambiguous or incomplete documentation is a recurrent bottleneck in radiation oncology workflows, leading to inefficiencies in communication and potential treatment delays. Large language models (LLMs) pose a solution to addressing these ambiguities without added burden to clinical staff. We aim to assess the effectiveness of Meta's open‐source Llama 3.3 model in using physician consultation notes to isolate and classify anatomical treatment sites and create helpful extractive summaries for each patient.
Methods
Semi‐structured interviews with five radiation therapists revealed that CT simulation orders lack the necessary details to acquire the appropriate image. A retrospective cohort of 100 patient notes was used for iterative prompt engineering. The final model was evaluated on an independent test cohort of 52 patient notes. The LLM's accuracy in identifying the treatment site was benchmarked against two human observers (a medical physicist and a physician) as well as the final delivered treatment plan (ground truth). The helpfulness and accuracy of the AI‐generated summaries were also rated by both observers on a 5‐point Likert scale.
Results
Llama 3.3 achieved a weighted accuracy of 94.2% [95%CI: 89.4%–98.1%] when compared to sites isolated by either observer. When compared to the sites isolated from the retrospectively delivered plans, the model reached a weighted accuracy of 92.3% [95% CI: 87.5%–97.1%]. The model classified the anatomical sites with a weighted accuracy of 96.2% [95%CI: 87.0% –98.9%]. The AI‐generated summaries were highly rated by both observers (Observer 1: 4.96 [95%CI: 4.87–5.00] and Observer 2: 4.58 [95% CI: 4.38–4.73]).
Conclusion
This pilot study provides foundational evidence that LLMs can classify data with high accuracy, achieve benchmarks comparable to human experts when isolating anatomical treatment sites, and produce clinically helpful summaries. Our results suggest that LLMs can be effectively integrated to streamline complex radiotherapy workflows in the clinic.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiation Oncology, University of California, Los Angeles, Los Angeles, California, USA.
ORCID-Profile: Klea Hoxha, Jack Neylon, Dylan P O'Connell
Zitieren
APA
Hoxha, K., Savjani, R. R., Neylon, J., Pham, J., Ariani, R., & O'Connell, D. P. (2026). Automated identification of radiotherapy treatment sites from unstructured physician notes. Journal of Applied Clinical Medical Physics, 27(4). https://doi.org/10.1002/acm2.70558
Vancouver
Hoxha K, Savjani RR, Neylon J, Pham J, Ariani R, O'Connell DP. Automated identification of radiotherapy treatment sites from unstructured physician notes. Journal of Applied Clinical Medical Physics. 2026;27(4). doi:10.1002/acm2.70558
BibTeX
@article{Hoxha2026,
author = {Hoxha, Klea and Savjani, Ricky R and Neylon, Jack and Pham, Jonathan and Ariani, Rojine and O'Connell, Dylan P},
title = {Automated identification of radiotherapy treatment sites from unstructured physician notes},
journal = {Journal of Applied Clinical Medical Physics},
year = {2026},
volume = {27},
number = {4},
doi = {10.1002/acm2.70558}
}
EUVIMED