Need for Objective Task-Based Evaluation of Image Segmentation Algorithms for Quantitative PET: A Study with ACRIN 6668/RTOG 0235 Multicenter Clinical Trial Data
Journal of Nuclear Medicine · 2024-03 · 65 (3) · 485-492
DOI: 10.2967/jnumed.123.266018 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Nuclear Medicine, online erschienen am 15.02.2024. Verfasst von Ziping Liu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Engineering, Washington University, St. Louis, Missouri. Hauptthemen laut MEDLINE: Carcinoma, Non-Small-Cell Lung, Lung Neoplasms. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIBIB NIH HHS.
Abstract (Originalsprache)
Reliable performance of PET segmentation algorithms on clinically relevant tasks is required for their clinical translation. However, these algorithms are typically evaluated using figures of merit (FoMs) that are not explicitly designed to correlate with clinical task performance. Such FoMs include the Dice similarity coefficient (DSC), the Jaccard similarity coefficient (JSC), and the Hausdorff distance (HD). The objective of this study was to investigate whether evaluating PET segmentation algorithms using these task-agnostic FoMs yields interpretations consistent with evaluation on clinically relevant quantitative tasks.
Methods
We conducted a retrospective study to assess the concordance in the evaluation of segmentation algorithms using the DSC, JSC, and HD and on the tasks of estimating the metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumors from PET images of patients with non-small cell lung cancer. The PET images were collected from the American College of Radiology Imaging Network 6668/Radiation Therapy Oncology Group 0235 multicenter clinical trial data. The study was conducted in 2 contexts: (1) evaluating conventional segmentation algorithms, namely those based on thresholding (SUVmax40% and SUVmax50%), boundary detection (Snakes), and stochastic modeling (Markov random field-Gaussian mixture model); (2) evaluating the impact of network depth and loss function on the performance of a state-of-the-art U-net-based segmentation algorithm.
Results
Evaluation of conventional segmentation algorithms based on the DSC, JSC, and HD showed that SUVmax40% significantly outperformed SUVmax50%. However, SUVmax40% yielded lower accuracy on the tasks of estimating MTV and TLG, with a 51% and 54% increase, respectively, in the ensemble normalized bias. Similarly, the Markov random field-Gaussian mixture model significantly outperformed Snakes on the basis of the task-agnostic FoMs but yielded a 24% increased bias in estimated MTV. For the U-net-based algorithm, our evaluation showed that although the network depth did not significantly alter the DSC, JSC, and HD values, a deeper network yielded substantially higher accuracy in the estimated MTV and TLG, with a decreased bias of 91% and 87%, respectively. Additionally, whereas there was no significant difference in the DSC, JSC, and HD values for different loss functions, up to a 73% and 58% difference in the bias of the estimated MTV and TLG, respectively, existed.
Conclusion
Evaluation of PET segmentation algorithms using task-agnostic FoMs could yield findings discordant with evaluation on clinically relevant quantitative tasks. This study emphasizes the need for objective task-based evaluation of image segmentation algorithms for quantitative PET.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biomedical Engineering, Washington University, St. Louis, Missouri.
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, Missouri; and.
- Alvin J. Siteman Cancer Center, Washington University School of Medicine, St. Louis, Missouri.
- Department of Biomedical Engineering, Washington University, St. Louis, Missouri; a.jha@wustl.edu.
Förderung
- NIBIB NIH HHS
Zitieren
APA
Liu, Z., Mhlanga, J. C., Xia, H., Siegel, B. A., & Jha, A. K. (2024). Need for Objective Task-Based Evaluation of Image Segmentation Algorithms for Quantitative PET: A Study with ACRIN 6668/RTOG 0235 Multicenter Clinical Trial Data. Journal of Nuclear Medicine, 65(3), 485-492. https://doi.org/10.2967/jnumed.123.266018
Vancouver
Liu Z, Mhlanga JC, Xia H, Siegel BA, Jha AK. Need for Objective Task-Based Evaluation of Image Segmentation Algorithms for Quantitative PET: A Study with ACRIN 6668/RTOG 0235 Multicenter Clinical Trial Data. Journal of Nuclear Medicine. 2024;65(3):485-492. doi:10.2967/jnumed.123.266018
BibTeX
@article{Liu2024,
author = {Liu, Ziping and Mhlanga, Joyce C and Xia, Huitian and Siegel, Barry A and Jha, Abhinav K},
title = {Need for Objective Task-Based Evaluation of Image Segmentation Algorithms for Quantitative PET: A Study with ACRIN 6668/RTOG 0235 Multicenter Clinical Trial Data},
journal = {Journal of Nuclear Medicine},
year = {2024},
volume = {65},
number = {3},
pages = {485--492},
doi = {10.2967/jnumed.123.266018}
}
EUVIMED