Assessment of the Diagnostic Performance of a Commercially Available Artificial Intelligence Algorithm for Risk Stratification of Thyroid Nodules on Ultrasound
Thyroid® · 2024
DOI: 10.1089/thy.2024.0410 ↗Kurz gefasst
Fachartikel in der Zeitschrift Thyroid®, 2024. Verfasst von Jeffrey Ashton und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Duke University Medical Center, Durham, North Carolina, USA. Hauptthemen laut MEDLINE: Thyroid Nodule, Ultrasonography, Artificial Intelligence, Algorithms. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 29 Literaturangaben.
Abstract (Originalsprache)
Background
Thyroid nodules are challenging to accurately characterize on ultrasound (US), though the emergence of risk stratification systems and more recently artificial intelligence (AI) algorithms has improved nodule classification. The purpose of this study was to evaluate the performance of a recent Food and Drug Administration (FDA)-cleared AI tool for detection of malignancy in thyroid nodules on US.
Methods
One year of consecutive thyroid US with ≥1 nodule from Duke University Hospital and its affiliate community hospital (649 nodules from 347 patients) were retrospectively evaluated. Included nodules had ground truth diagnoses by surgical pathology, fine needle aspiration (FNA), or three-year follow-up US showing stability. An FDA-cleared AI tool (Koios DS Thyroid) analyzed each nodule to generate (i) American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) descriptors, scores, and follow-up recommendations and (ii) an AI-adapter score to further adjust risk assessments and recommendations. Four groups were then compared: (i) Koios with AI-adapter, (ii) Koios without AI-adapter, (iii) clinical radiology report, and (iv) radiology report combined with AI-adapter. Performance of the final recommendations (FNA or no FNA) was determined based on ground truth, and comparison between the four groups was made using sensitivity, specificity, and receiver-operating-curve analysis.
Results
Of 649 nodules, 32 were malignant and 617 were benign. Performance of Koios with AI-adapter enabled was similar to radiologists (area under the curve [AUC] 0.70 for both, [CI 0.60–0.81] and [0.60–0.79], respectively). Koios with AI-adapter had improved specificity compared to radiologists (0.63 [CI: 0.59–0.67] versus 0.43 [CI: 0.38–0.48]) but decreased sensitivity (0.69 [CI: 0.50–0.83) versus 0.81 [CI: 0.61, 0.92]). Highest performance was seen when the radiology interpretation was combined with the AI-adapter (AUC 0.76, [CI: 0.67–0.85]). Combined with the AI-adapter, radiologist specificity improved from 0.43 ([CI: 0.38–0.48]) to 0.53 ([CI: 0.49–0.58]) (McNemar’s test p < 0.001), resulting in 17% fewer FNA recommendations, with unchanged sensitivity (0.81, p = 1).
Conclusion
Koios DS demonstrated standalone performance similar to radiologists, though with lower sensitivity and higher specificity. Performance was best when radiologist interpretations were combined with the AI-adapter component, with improved specificity and reduced unnecessary FNA recommendations.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiology, Duke University Medical Center, Durham, North Carolina, USA.
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, North Carolina, USA.
Zitieren
APA
Ashton, J., Morrison, S., Erkanli, A., & Wildman-Tobriner, B. (2024). Assessment of the Diagnostic Performance of a Commercially Available Artificial Intelligence Algorithm for Risk Stratification of Thyroid Nodules on Ultrasound. Thyroid®. https://doi.org/10.1089/thy.2024.0410
Vancouver
Ashton J, Morrison S, Erkanli A, Wildman-Tobriner B. Assessment of the Diagnostic Performance of a Commercially Available Artificial Intelligence Algorithm for Risk Stratification of Thyroid Nodules on Ultrasound. Thyroid®. 2024. doi:10.1089/thy.2024.0410
BibTeX
@article{Ashton2024,
author = {Ashton, Jeffrey and Morrison, Samantha and Erkanli, Alaattin and Wildman-Tobriner, Benjamin},
title = {Assessment of the Diagnostic Performance of a Commercially Available Artificial Intelligence Algorithm for Risk Stratification of Thyroid Nodules on Ultrasound},
journal = {Thyroid®},
year = {2024},
doi = {10.1089/thy.2024.0410}
}
EUVIMED