EUVIMEDEUVIMED entdecken
Open Access

Understanding Transformer-Based Classifications of Medical Text Using a Large Language Model for the Attribution of Feature Importance: Proof-of-Concept Algorithm Development and Validation Study

Fangwen Zhou iD, Ashirbani Saha iD, Muhammad Afzal iD, Rick Parrish iD, R Brian Haynes iD, Alfonso Iorio iD, Cynthia Lokker iD

JMIR Medical Informatics · 2026

DOI: 10.2196/81644 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Medical Informatics, 2026. Verfasst von Fangwen Zhou und 6 weiteren Autor:innen. Beteiligte Einrichtung: Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, 1280 …. Hauptthemen laut MEDLINE: Classification Algorithms, Large Language Models. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Deep learning, particularly encoder-only transformer architectures, has demonstrated excellent performance in biomedical literature classification, facilitating evidence-based medicine, and knowledge synthesis. However, the opacity of these models’ decision-making processes limits their clinical interpretability, trustworthiness, and widespread adoption. Traditional explainable artificial intelligence methods, such as Shapley Additive Explanations (SHAP) and integrated gradients (IG), address this issue but often incur substantial computational overhead for text classification. Generative large language models may offer a novel approach to generating interpretable, context-aware explanations as autonomous agents.

Objective

As a proof-of-concept, the study aimed to investigate the effectiveness of GPT-4o as a standalone, end-to-end perturbation-based explainer for a BioLinkBERT text classifier. We compared its explanations against the SHAP partition explainer and IG as established baselines in terms of explanation faithfulness and semantic alignment.

Methods

A stratified sample of 200 studies from the McMaster Premium Literature Service (PLUS) and Clinical Hedges databases was classified by a fine-tuned BioLinkBERT model for methodological rigor. The sampling specifically over-represented difficult, low-confidence predictions to rigorously test the explainers, with an equal number of studies sampled from each probability decile predicted by BioLinkBERT. GPT-4o, SHAP, and IG generated token-level feature attributions across a robust feature space of 80,901 tokens. GPT-based explanations were derived through a sophisticated, iterative masking perturbation workflow under 2 prompting schemes (token indices vs explicit subword tokens). Explanations were evaluated using a rank-based, modified area over the perturbation curve (AOPC), pairwise correlation analyses, and qualitative assessment of feature importance.

Results

Among the 200 studies, 80,901 tokens were included, and feature attributions were generated by the 4 explainers (6369 unique tokens). SHAP (AOPC 0.222, 95% CI 0.200-0.244) and IG (AOPC 0.225, 95% CI 0.202-0.247) provided consistent explanations, effectively identifying tokens relevant to study rigor (eg, “randomized” and “blind”). In contrast, despite evaluating a larger perturbation space, the GPT-4o prompting schemes did not achieve comparable faithfulness (AOPC 0.025-0.029) and produced divergent token attributions. Correlation analysis demonstrated moderate alignment between SHAP and IG (Pearson r= 0.367), whereas GPT-4o exhibited limited correlation (Pearson r ≤0.032) with the established baselines. Sensitivity analyses isolating only correctly classified instances yielded similar trends. Additionally, the iterative application programming interface calls required for GPT made it significantly more computationally intensive and costly to execute, whereas IG was the most temporally efficient.

Conclusions

Despite their advanced contextual capabilities, current generative large language models are limited when deployed as standalone perturbation explainers. The findings reveal that GPT-4o struggles to accurately synthesize mathematical feature importance through iterative masking, lacking the reliability of traditional explainable artificial intelligence frameworks. Future research could build upon this work and investigate specialized prompt engineering, whole-word recombination strategies, and hybrid frameworks.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Classification AlgorithmsLarge Language ModelsHumansGenerative Artificial IntelligenceNatural Language ProcessingProof of Concept Study

Stichwörter der Autor:innen

GPTSHAPShapley Additive Explanationsartificial intelligencedeep learningexplainable artificial intelligencefeature attributionintegrated gradientsnatural language processing

Autor:innen und Institutionen

  • Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, 1280 Main Street West, Hamilton, ON, L8S 4L8, Canada, 1 905-525-9140 ext 22…
  • Department of Oncology, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada.
  • Department of Computer Science, Birmingham City University, Birmingham, United Kingdom.
  • Department of Medicine, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada.

ORCID-Profile: Fangwen Zhou, Ashirbani Saha, Muhammad Afzal, Rick Parrish, R Brian Haynes, Alfonso Iorio, Cynthia Lokker

Zitieren

APA

Zhou, F., Saha, A., Afzal, M., Parrish, R., Haynes, R. B., Iorio, A., & Lokker, C. (2026). Understanding Transformer-Based Classifications of Medical Text Using a Large Language Model for the Attribution of Feature Importance: Proof-of-Concept Algorithm Development and Validation Study. JMIR Medical Informatics. https://doi.org/10.2196/81644

Vancouver

Zhou F, Saha A, Afzal M, Parrish R, Haynes RB, Iorio A, et al. Understanding Transformer-Based Classifications of Medical Text Using a Large Language Model for the Attribution of Feature Importance: Proof-of-Concept Algorithm Development and Validation Study. JMIR Medical Informatics. 2026. doi:10.2196/81644

BibTeX
@article{Zhou2026,
  author = {Zhou, Fangwen and Saha, Ashirbani and Afzal, Muhammad and Parrish, Rick and Haynes, R Brian and Iorio, Alfonso and Lokker, Cynthia},
  title = {Understanding Transformer-Based Classifications of Medical Text Using a Large Language Model for the Attribution of Feature Importance: Proof-of-Concept Algorithm Development and Validation Study},
  journal = {JMIR Medical Informatics},
  year = {2026},
  doi = {10.2196/81644}
}
Kostenlos starten →