Gross failure rates and failure modes for a commercial AI‐based auto‐segmentation algorithm in head and neck cancer patients
Journal of Applied Clinical Medical Physics · 2024
DOI: 10.1002/acm2.14273 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Applied Clinical Medical Physics, 2024. Verfasst von Simon W P Temple und 1 weiteren Autor:innen. Beteiligte Einrichtung: Medical Physics Department, The Clatterbridge Cancer Centre NHS Foundation Trust, Liverpool, UK. Hauptthemen laut MEDLINE: Head and Neck Neoplasms, Radiotherapy Planning, Computer-Assisted, Algorithms, Organs at Risk. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Themen: Onkologische Pflege
Abstract (Originalsprache)
Purpose
Artificial intelligence (AI) based commercial software can be used to automatically delineate organs at risk (OAR), with potential for efficiency savings in the radiotherapy treatment planning pathway, and reduction of inter‐ and intra‐observer variability. There has been little research investigating gross failure rates and failure modes of such systems.
Method
50 head and neck (H&N) patient data sets with “gold standard” contours were compared to AI‐generated contours to produce expected mean and standard deviation values for the Dice Similarity Coefficient (DSC), for four common H&N OARs (brainstem, mandible, left and right parotid). An AI‐based commercial system was applied to 500 H&N patients. AI‐generated contours were compared to manual contours, outlined by an expert human, and a gross failure was set at three standard deviations below the expected mean DSC. Failures were inspected to assess reason for failure of the AI‐based system with failures relating to suboptimal manual contouring censored. True failures were classified into 4 sub‐types (setup position, anatomy, image artefacts and unknown).
Results
There were 24 true failures of the AI‐based commercial software, a gross failure rate of 1.2%. Fifteen failures were due to patient anatomy, four were due to dental image artefacts, three were due to patient position and two were unknown. True failure rates by OAR were 0.4% (brainstem), 2.2% (mandible), 1.4% (left parotid) and 0.8% (right parotid).
Conclusion
True failures of the AI‐based system were predominantly associated with a non‐standard element within the CT scan. It is likely that these non‐standard elements were the reason for the gross failure, and suggests that patient datasets used to train the AI model did not contain sufficient heterogeneity of data. Regardless of the reasons for failure, the true failure rate for the AI‐based system in the H&N region for the OARs investigated was low (∼1%).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Medical Physics Department, The Clatterbridge Cancer Centre NHS Foundation Trust, Liverpool, UK.
- Department of Physics, University of Liverpool, Liverpool, UK.
Zitieren
APA
Temple, S. W. P., & Rowbottom, C. G. (2024). Gross failure rates and failure modes for a commercial AI‐based auto‐segmentation algorithm in head and neck cancer patients. Journal of Applied Clinical Medical Physics. https://doi.org/10.1002/acm2.14273
Vancouver
Temple SWP, Rowbottom CG. Gross failure rates and failure modes for a commercial AI‐based auto‐segmentation algorithm in head and neck cancer patients. Journal of Applied Clinical Medical Physics. 2024. doi:10.1002/acm2.14273
BibTeX
@article{Temple2024,
author = {Temple, Simon W P and Rowbottom, Carl G},
title = {Gross failure rates and failure modes for a commercial AI‐based auto‐segmentation algorithm in head and neck cancer patients},
journal = {Journal of Applied Clinical Medical Physics},
year = {2024},
doi = {10.1002/acm2.14273}
}
EUVIMED