EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Prompting and Fine-Tuning Large Language Models for Parkinson Disease Diagnosis: Comparative Evaluation Study Using the PPMI Structured Dataset

Hyun-Ji Shin iD, Young Jin Jeong iD, Sungmin Jun iD, Do-Young Kang iD

JMIR Medical Informatics · 15.01.2026 · 14 · e77561

DOI: 10.2196/77561 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Medical Informatics, online erschienen am 15.01.2026. Verfasst von Hyun-Ji Shin und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Data Sciences Convergence, Graduate School, Dong-A University, Busan, Republic of Korea. Hauptthemen laut MEDLINE: Large Language Models, Parkinson Disease. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Parkinson disease (PD) presents diagnostic challenges due to its heterogeneous motor and nonmotor manifestations. Traditional machine learning (ML) approaches have been evaluated on structured clinical variables. However, the diagnostic utility of large language models (LLMs) using natural language representations of structured clinical data remains underexplored.

Objective

This study aimed to evaluate the diagnostic classification performance of multiple LLMs using natural language prompts derived from structured clinical data and to compare their performance with traditional ML baselines.

Methods

We reformatted structured clinical variables from the Parkinson’s Progression Markers Initiative (PPMI) dataset into natural language prompts and used them as inputs for several LLMs. Variables with high multicollinearity were removed, and the top 10 features were selected using Shapley additive explanations (SHAP)–based feature ranking. LLM performance was examined across few-shot prompting, dual-output prompting that additionally generated post hoc explanatory text as an exploratory component, and supervised fine-tuning. Logistic regression (LR) and support vector machine (SVM) classifiers served as ML baselines. Model performance was evaluated using F1-scores on both the test set and a temporally independent validation set (temporal validation set) of limited size, and repeated output generation was carried out to assess stability.

Results

On the test set of 122 participants, LR and SVM trained on the 10 SHAP-selected clinical variables each achieved a macro-averaged F1-score of 0.960 (accuracy 0.975). LLMs receiving natural language prompts derived from the same variables reached comparable performance, with the best few-shot configurations achieving macro-averaged F1-scores of 0.987 (accuracy 0.992). In the temporal validation set of 31 participants, LR maintained a macro-averaged F1-score of 0.903, whereas SVM showed substantial performance degradation. In contrast, multiple LLMs sustained high diagnostic performance, reaching macro-averaged F1-scores up to 0.968 and high recall for PD. Repeated output generation across LLM conditions produced generally stable predictions, with rare variability observed across runs. Under dual-output prompting, diagnostic performance showed a reduction relative to few-shot prompting while remaining generally stable. Supervised fine-tuning of lightweight models improved stability and enabled GPT-4o-mini to achieve a macro-averaged F1-score of 0.987 on the test set, with uniformly correct predictions observed in the small temporal validation set, which should be interpreted cautiously given the limited sample size and exploratory nature of the evaluation.

Conclusions

This study provides an exploratory benchmark of how modern LLMs process structured clinical variables in natural language form. While several models achieved diagnostic performance comparable to LR across both the test and temporal validation datasets, their outputs were sensitive to prompting formats, model choice, and class distributions. Occasional variability across repeated output generations reflected the stochastic nature of LLMs, and lightweight models required supervised fine-tuning for stable generalization. These findings highlight the capabilities and limitations of current LLMs in handling tabular clinical information and underscore the need for cautious application and further investigation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsParkinson DiseaseHumansSupport Vector MachineMachine LearningPredictive Learning Models

Stichwörter der Autor:innen

ClaudeGPTGeminiLLaMAPPMIParkinson diseaseParkinson’s Progression Markers Initiativediagnostic classificationfine-tuninglarge language modelsprompt engineering

Autor:innen und Institutionen

  • Department of Data Sciences Convergence, Graduate School, Dong-A University, Busan, Republic of Korea.
  • Institute of Convergence Bio-Health, Dong-A University, Busan, Republic of Korea.
  • Department of Nuclear Medicine, Dong-A University Hospital, Busan, Republic of Korea.
  • Department of Nuclear Medicine, College of Medicine, Dong-A University, Busan, Republic of Korea.

ORCID-Profile: Hyun-Ji Shin, Young Jin Jeong, Sungmin Jun, Do-Young Kang

Zitieren

APA

Shin, H. J., Jeong, Y. J., Jun, S., & Kang, D. Y. (2026). Prompting and Fine-Tuning Large Language Models for Parkinson Disease Diagnosis: Comparative Evaluation Study Using the PPMI Structured Dataset. JMIR Medical Informatics, 14, e77561. https://doi.org/10.2196/77561

Vancouver

Shin HJ, Jeong YJ, Jun S, Kang DY. Prompting and Fine-Tuning Large Language Models for Parkinson Disease Diagnosis: Comparative Evaluation Study Using the PPMI Structured Dataset. JMIR Medical Informatics. 2026;14:e77561. doi:10.2196/77561

BibTeX
@article{Shin2026,
  author = {Shin, Hyun-Ji and Jeong, Young Jin and Jun, Sungmin and Kang, Do-Young},
  title = {Prompting and Fine-Tuning Large Language Models for Parkinson Disease Diagnosis: Comparative Evaluation Study Using the PPMI Structured Dataset},
  journal = {JMIR Medical Informatics},
  year = {2026},
  volume = {14},
  pages = {e77561},
  doi = {10.2196/77561}
}
Kostenlos starten →