EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY-NC 4.0

Evaluating the efficacy of few‐shot learning for GPT‐4Vision in neurodegenerative disease histopathology: A comparative analysis with convolutional neural network model

Daisuke Ono iD, Dennis W Dickson iD, Shunsuke Koga iD

Neuropathology and Applied Neurobiology · 2024-08 · 50 (4)

DOI: 10.1111/nan.12997 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Neuropathology and Applied Neurobiology, online erschienen am 15.07.2024. Verfasst von Daisuke Ono und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Neuroscience, Mayo Clinic, Jacksonville, Florida, USA. Hauptthemen laut MEDLINE: Neural Networks, Computer, Neurodegenerative Diseases. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aims

Recent advances in artificial intelligence, particularly with large language models like GPT‐4Vision (GPT‐4V)—a derivative feature of ChatGPT—have expanded the potential for medical image interpretation. This study evaluates the accuracy of GPT‐4V in image classification tasks of histopathological images and compares its performance with a traditional convolutional neural network (CNN).

Methods

We utilised 1520 images, including haematoxylin and eosin staining and tau immunohistochemistry, from patients with various neurodegenerative diseases, such as Alzheimer's disease (AD), progressive supranuclear palsy (PSP) and corticobasal degeneration (CBD). We assessed GPT‐4V's performance using multi‐step prompts to determine how textual context influences image interpretation. We also employed few‐shot learning to enhance improvements in GPT‐4V's diagnostic performance in classifying three specific tau lesions—astrocytic plaques, neuritic plaques and tufted astrocytes—and compared the outcomes with the CNN model YOLOv8.

Results

GPT‐4V accurately recognised staining techniques and tissue origin but struggled with specific lesion identification. The interpretation of images was notably influenced by the provided textual context, which sometimes led to diagnostic inaccuracies. For instance, when presented with images of the motor cortex, the diagnosis shifted inappropriately from AD to CBD or PSP. However, few‐shot learning markedly improved GPT‐4V's diagnostic capabilities, enhancing accuracy from 40% in zero‐shot learning to 90% with 20‐shot learning, matching the performance of YOLOv8, which required 100‐shot learning to achieve the same accuracy.

Conclusions

Although GPT‐4V faces challenges in independently interpreting histopathological images, few‐shot learning significantly improves its performance. This approach is especially promising for neuropathology, where acquiring extensive labelled datasets is often challenging.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Neural Networks, ComputerNeurodegenerative DiseasesHumansImage Interpretation, Computer-AssistedAlzheimer Disease

Stichwörter der Autor:innen

Alzheimer's diseaseChatGPTGPT‐4Visionconvolutional neural networkdigital pathologyfew‐shot learninglarge language modelneuropathology

Autor:innen und Institutionen

  • Department of Neuroscience, Mayo Clinic, Jacksonville, Florida, USA.
  • Department of Pathology and Laboratory Medicine, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA.

ORCID-Profile: Daisuke Ono, Dennis W Dickson, Shunsuke Koga

Zitieren

APA

Ono, D., Dickson, D. W., & Koga, S. (2024). Evaluating the efficacy of few‐shot learning for GPT‐4Vision in neurodegenerative disease histopathology: A comparative analysis with convolutional neural network model. Neuropathology and Applied Neurobiology, 50(4). https://doi.org/10.1111/nan.12997

Vancouver

Ono D, Dickson DW, Koga S. Evaluating the efficacy of few‐shot learning for GPT‐4Vision in neurodegenerative disease histopathology: A comparative analysis with convolutional neural network model. Neuropathology and Applied Neurobiology. 2024;50(4). doi:10.1111/nan.12997

BibTeX
@article{Ono2024,
  author = {Ono, Daisuke and Dickson, Dennis W and Koga, Shunsuke},
  title = {Evaluating the efficacy of few‐shot learning for GPT‐4Vision in neurodegenerative disease histopathology: A comparative analysis with convolutional neural network model},
  journal = {Neuropathology and Applied Neurobiology},
  year = {2024},
  volume = {50},
  number = {4},
  doi = {10.1111/nan.12997}
}
Kostenlos starten →