EUVIMEDEUVIMED entdecken
Open Access · CC0

Assessing generalizability of an AI-based visual test for cervical cancer screening

Syed Rakin Ahmed iD, Didem Egemen, Brian Befano, Ana Cecilia Rodriguez, Jose Jeronimo, Kanan Desai iD, Carolina Teran iD, Karla Alfaro, Joel Fokom-Domgue iD, Kittipat Charoenkwan, Chemtai Mungo iD, Rebecca Luckett und 7 weitere

PLOS Digital Health · 02.10.2024 · 3 (10) · e0000364

DOI: 10.1371/journal.pdig.0000364 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Digital Health, online erschienen am 02.10.2024. Verfasst von Syed Rakin Ahmed und 18 weiteren Autor:innen. Beteiligte Einrichtung: Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of…. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC0). Gefördert von: NCI NIH HHS, Cancer Moonshot, National Cancer Institute.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A number of challenges hinder artificial intelligence (AI) models from effective clinical translation. Foremost among these challenges is the lack of generalizability, which is defined as the ability of a model to perform well on datasets that have different characteristics from the training data. We recently investigated the development of an AI pipeline on digital images of the cervix, utilizing a multi-heterogeneous dataset of 9,462 women (17,013 images) and a multi-stage model selection and optimization approach, to generate a diagnostic classifier able to classify images of the cervix into “normal”, “indeterminate” and “precancer/cancer” (denoted as “precancer+”) categories. In this work, we investigate the performance of this multiclass classifier on external data not utilized in training and internal validation, to assess the generalizability of the classifier when moving to new settings. We assessed both the classification performance and repeatability of our classifier model across the two axes of heterogeneity present in our dataset: image capture device and geography, utilizing both out-of-the-box inference and retraining with external data. Our results demonstrate that device-level heterogeneity affects our model performance more than geography-level heterogeneity. Classification performance of our model is strong on images from a new geography without retraining, while incremental retraining with inclusion of images from a new device progressively improves classification performance on that device up to a point of saturation. Repeatability of our model is relatively unaffected by data heterogeneity and remains strong throughout. Our work supports the need for optimized retraining approaches that address data heterogeneity (e.g., when moving to a new device) to facilitate effective use of AI models in new settings.

Autor:innen und Institutionen

  • Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, United States of America.
  • Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Massachusetts, United States of America.
  • Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America.
  • Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, New Hampshire, United States of America.
  • Clinical Epidemiology Unit, Clinical Genetics Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Bethesda, Maryland, United States of America.
  • Information Management Services, Calverton, Maryland, United States of America.

ORCID-Profile: Syed Rakin Ahmed, Kanan Desai, Carolina Teran, Joel Fokom-Domgue, Chemtai Mungo, Julia C Gage

Förderung

  • NCI NIH HHS
  • Cancer Moonshot
  • National Cancer Institute

Zitieren

APA

Ahmed, S. R., Egemen, D., Befano, B., Rodriguez, A. C., Jeronimo, J., Desai, K., Teran, C., Alfaro, K., Fokom-Domgue, J., Charoenkwan, K., Mungo, C., Luckett, R., Saidu, R., Raiol, T., Ribeiro, A., Gage, J. C., de Sanjose, S., Kalpathy-Cramer, J., & Schiffman, M. (2024). Assessing generalizability of an AI-based visual test for cervical cancer screening. PLOS Digital Health, 3(10), e0000364. https://doi.org/10.1371/journal.pdig.0000364

Vancouver

Ahmed SR, Egemen D, Befano B, Rodriguez AC, Jeronimo J, Desai K, et al. Assessing generalizability of an AI-based visual test for cervical cancer screening. PLOS Digital Health. 2024;3(10):e0000364. doi:10.1371/journal.pdig.0000364

BibTeX
@article{Ahmed2024,
  author = {Ahmed, Syed Rakin and Egemen, Didem and Befano, Brian and Rodriguez, Ana Cecilia and Jeronimo, Jose and Desai, Kanan and Teran, Carolina and Alfaro, Karla and Fokom-Domgue, Joel and Charoenkwan, Kittipat and Mungo, Chemtai and Luckett, Rebecca and Saidu, Rakiya and Raiol, Taina and Ribeiro, Ana and Gage, Julia C and de Sanjose, Silvia and Kalpathy-Cramer, Jayashree and Schiffman, Mark},
  title = {Assessing generalizability of an AI-based visual test for cervical cancer screening},
  journal = {PLOS Digital Health},
  year = {2024},
  volume = {3},
  number = {10},
  pages = {e0000364},
  doi = {10.1371/journal.pdig.0000364}
}
Kostenlos starten →