Evaluation of multiple-vendor AI autocontouring solutions
Radiation Oncology · 31.05.2024 · 19 (1)
DOI: 10.1186/s13014-024-02451-4 ↗Kurz gefasst
Evaluationsstudie in der Zeitschrift Radiation Oncology, online erschienen am 31.05.2024. Verfasst von Lee Goddard und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, Montefiore Medical Center, Bronx, NY, 10467, USA. Hauptthemen laut MEDLINE: Artificial Intelligence, Tomography, X-Ray Computed, Radiotherapy Planning, Computer-Assisted. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Multiple artificial intelligence (AI)-based autocontouring solutions have become available, each promising high accuracy and time savings compared with manual contouring. Before implementing AI-driven autocontouring into clinical practice, three commercially available CT-based solutions were evaluated. Materials and methods The following solutions were evaluated in this work: MIM-ProtégéAI+ (MIM), Radformation-AutoContour (RAD), and Siemens-DirectORGANS (SIE). Sixteen organs were identified that could be contoured by all solutions. For each organ, ten patients that had manually generated contours approved by the treating physician (AP) were identified, totaling forty-seven different patients. CT scans in the supine position were acquired using a Siemens-SOMATOMgo 64-slice helical scanner and used to generate autocontours. Physician scoring of contour accuracy was performed by at least three physicians using a five-point Likert scale. Dice similarity coefficient (DSC), Hausdorff distance (HD) and mean distance to agreement (MDA) were calculated comparing AI contours to “ground truth” AP contours.
Results
The average physician score ranged from 1.00, indicating that all physicians reviewed the contour as clinically acceptable with no modifications necessary, to 3.70, indicating changes are required and that the time taken to modify the structures would likely take as long or longer than manually generating the contour. When averaged across all sixteen structures, the AP contours had a physician score of 2.02, MIM 2.07, RAD 1.96 and SIE 1.99. DSC ranged from 0.37 to 0.98, with 41/48 (85.4%) contours having an average DSC ≥ 0.7. Average HD ranged from 2.9 to 43.3 mm. Average MDA ranged from 0.6 to 26.1 mm.
Conclusions
The results of our comparison demonstrate that each vendor’s AI contouring solution exhibited capabilities similar to those of manual contouring. There were a small number of cases where unusual anatomy led to poor scores with one or more of the solutions. The consistency and comparable performance of all three vendors’ solutions suggest that radiation oncology centers can confidently choose any of the evaluated solutions based on individual preferences, resource availability, and compatibility with their existing clinical workflows. Although AI-based contouring may result in high-quality contours for the majority of patients, a minority of patients require manual contouring and more in-depth physician review.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiation Oncology, Montefiore Medical Center, Bronx, NY, 10467, USA.
- Albert Einstein College of Medicine, Bronx, NY, 10461, USA.
- Department of Radiation Oncology, Montefiore Medical Center, Bronx, NY, 10467, USA. wolfgang.tome@einsteinmed.edu.
- Albert Einstein College of Medicine, Bronx, NY, 10461, USA. wolfgang.tome@einsteinmed.edu.
- Division of Medical Physics, Albert Einstein College of Medicine, 1300 Morris Park Ave, Block Building Room 106, Bronx, NY, 10461, USA. wolfgang.tome@einsteinmed.edu.
Zitieren
APA
Goddard, L., Velten, C., Tang, J., Skalina, K. A., Boyd, R., Martin, W., Basavatia, A., Garg, M., & Tomé, W. A. (2024). Evaluation of multiple-vendor AI autocontouring solutions. Radiation Oncology, 19(1). https://doi.org/10.1186/s13014-024-02451-4
Vancouver
Goddard L, Velten C, Tang J, Skalina KA, Boyd R, Martin W, et al. Evaluation of multiple-vendor AI autocontouring solutions. Radiation Oncology. 2024;19(1). doi:10.1186/s13014-024-02451-4
BibTeX
@article{Goddard2024,
author = {Goddard, Lee and Velten, Christian and Tang, Justin and Skalina, Karin A and Boyd, Robert and Martin, William and Basavatia, Amar and Garg, Madhur and Tomé, Wolfgang A},
title = {Evaluation of multiple-vendor AI autocontouring solutions},
journal = {Radiation Oncology},
year = {2024},
volume = {19},
number = {1},
doi = {10.1186/s13014-024-02451-4}
}
EUVIMED