EUVIMEDEUVIMED entdecken
Vergleichsstudie

Consistency of convolutional neural networks in dermoscopic melanoma recognition: A prospective real‐world study about the pitfalls of augmented intelligence

E V Goessinger iD, S E Cerminara iD, A M Mueller, P Gottfrois, S Huber, M Amaral, F Wenz, L Kostner, L Weiss, M Kunz, J-T Maul iD, S Wespi und 6 weitere

Journal of the European Academy of Dermatology and Venereology · 2024

DOI: 10.1111/jdv.19777 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Journal of the European Academy of Dermatology and Venereology, 2024. Verfasst von E V Goessinger und 17 weiteren Autor:innen. Beteiligte Einrichtung: Department of Dermatology, University Hospital Basel, Basel, Switzerland. Hauptthemen laut MEDLINE: Melanoma, Dermoscopy, Skin Neoplasms, Neural Networks, Computer. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 21 Literaturangaben. Gefördert von: Research Foundation of the University Basel, University Hospital Basel, Voluntary Academic Society Basel.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Deep‐learning convolutional neural networks (CNNs) have outperformed even experienced dermatologists in dermoscopic melanoma detection under controlled conditions. It remains unexplored how real‐world dermoscopic image transformations affect CNN robustness.

Objectives

To investigate the consistency of melanoma risk assessment by two commercially available CNNs to help formulate recommendations for current clinical use.

Methods

A comparative cohort study was conducted from January to July 2022 at the Department of Dermatology, University Hospital Basel. Five dermoscopic images of 116 different lesions on the torso of 66 patients were captured consecutively by the same operator without deliberate rotation. Classification was performed by two CNNs (CNN‐1/CNN‐2). Lesions were divided into four subgroups based on their initial risk scoring and clinical dignity assessment. Reliability was assessed by variation and intraclass correlation coefficients. Excisions were performed for melanoma suspicion or two consecutively elevated CNN risk scores, and benign lesions were confirmed by expert consensus ( n = 3).

Results

117 repeated image series of 116 melanocytic lesions (2 melanomas, 16 dysplastic naevi, 29 naevi, 1 solar lentigo, 1 suspicious and 67 benign) were classified. CNN‐1 demonstrated superior measurement repeatability for clinically benign lesions with an initial malignant risk score (mean variation coefficient (mvc): CNN‐1: 49.5(±34.3)%; CNN‐2: 71.4(±22.5)%; p = 0.03), while CNN‐2 outperformed for clinically benign lesions with benign scoring (mvc: CNN‐1: 49.7(±22.7)%; CNN‐2: 23.8(±29.3)%; p = 0.002). Both systems exhibited lowest score consistency for lesions with an initial malignant risk score and benign assessment. In this context, averaging three initial risk scores achieved highest sensitivity of dignity assessment (CNN‐1: 94%; CNN‐2: 89%). Intraclass correlation coefficients indicated ‘moderate’‐to‐‘good’ reliability for both systems (CNN‐1: 0.80, 95% CI:0.71–0.87, p < 0.001; CNN‐2: 0.67, 95% CI:0.55–0.77, p < 0.001).

Conclusions

Potential user‐induced image changes can significantly influence CNN classification. For clinical application, we recommend using the average of three initial risk scores. Furthermore, we advocate for CNN robustness optimization by cross‐validation with repeated image sets. Trial Registration ClinicalTrials.gov (NCT04605822).

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

MelanomaDermoscopySkin NeoplasmsNeural Networks, ComputerHumansProspective StudiesMaleFemaleMiddle AgedReproducibility of ResultsAdultAgedRisk AssessmentDeep LearningDysplastic Nevus Syndrome

Autor:innen und Institutionen

  • Department of Dermatology, University Hospital Basel, Basel, Switzerland.
  • Department of Dermatology, University Hospital Zurich, Zurich, Switzerland.

ORCID-Profile: E V Goessinger, S E Cerminara, J-T Maul, A A Navarini, L V Maul

Förderung

  • Research Foundation of the University Basel
  • University Hospital Basel
  • Voluntary Academic Society Basel
  • ProPatient Reseach Foundation University Basel
  • Universitätsspital Basel

Zitieren

APA

Goessinger, E. V., Cerminara, S. E., Mueller, A. M., Gottfrois, P., Huber, S., Amaral, M., Wenz, F., Kostner, L., Weiss, L., Kunz, M., Maul, J. T., Wespi, S., Broman, E., Kaufmann, S., Patpanathapillai, V., Treyer, I., Navarini, A. A., & Maul, L. V. (2024). Consistency of convolutional neural networks in dermoscopic melanoma recognition: A prospective real‐world study about the pitfalls of augmented intelligence. Journal of the European Academy of Dermatology and Venereology. https://doi.org/10.1111/jdv.19777

Vancouver

Goessinger EV, Cerminara SE, Mueller AM, Gottfrois P, Huber S, Amaral M, et al. Consistency of convolutional neural networks in dermoscopic melanoma recognition: A prospective real‐world study about the pitfalls of augmented intelligence. Journal of the European Academy of Dermatology and Venereology. 2024. doi:10.1111/jdv.19777

BibTeX
@article{Goessinger2024,
  author = {Goessinger, E V and Cerminara, S E and Mueller, A M and Gottfrois, P and Huber, S and Amaral, M and Wenz, F and Kostner, L and Weiss, L and Kunz, M and Maul, J-T and Wespi, S and Broman, E and Kaufmann, S and Patpanathapillai, V and Treyer, I and Navarini, A A and Maul, L V},
  title = {Consistency of convolutional neural networks in dermoscopic melanoma recognition: A prospective real‐world study about the pitfalls of augmented intelligence},
  journal = {Journal of the European Academy of Dermatology and Venereology},
  year = {2024},
  doi = {10.1111/jdv.19777}
}
Kostenlos starten →