EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Med-LLaMA3: Advancing Medical Question-Answering Through Parameter-Efficient Fine-Tuning of Large Language Models

Mohamed Ahmed Abo El-Enen, Sally S. Ismail, Taymoor Mohamed Nazmy

Applied Sciences · 17.06.2026 · 16 (12) · 6158

DOI: 10.3390/app16126158 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 17.06.2026. Verfasst von Mohamed Ahmed Abo El-Enen und 2 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Despite recent advances, medical question answering systems still struggle with domain-specific reasoning and data efficiency. This paper presents Med-LLaMA3, a family of medical large language models developed by parameter-efficient fine-tuning of the LLaMA-3.1 (8 billion) and LLaMA-3.2 (1 and 3 billion) architectures using quantized low-rank adaptation (QLoRA) and low-rank adaptation (LoRA) with 4-bit quantization. Beyond model training, this work contributes the following: (1) a formalized dataset curation taxonomy (source type × clinical granularity × task format) with a source-category ablation confirming that the multi-source combination drives benchmark gains beyond any single category; (2) a systematic characterization of low-rank-adaptation rank-scaling behavior for the LLaMA-3 family in the medical domain (monotonic improvement up to rank 128, with no observed plateau); and (3) statistically validated comparisons using McNemar’s test and 95% bootstrap confidence intervals. We curated a medical instruction dataset of over 1.5 million samples spanning medical examinations, clinical dialogues, and biomedical literature. Our approach trains only ∼4% of the base model’s parameters and, consistent with prior studies of parameter-efficient methods in the medical domain, achieves performance comparable to full fine-tuning at a fraction of the memory footprint. Evaluated with five in-context examples per prompt, the 8-billion-parameter model attains a mean accuracy of 75.71% across the eight medical-domain subsets of the Massive Multitask Language Understanding benchmark; improvements over the unmodified LLaMA-3.1-8B-Instruct baseline are statistically significant on the medical multiple-choice benchmark MedMCQA and, after Bonferroni correction across the eight subsets, on three subsets (Clinical Knowledge, Medical Genetics, and Nutrition), with two further subsets being significant only before correction. A structured named-entity-recognition evaluation on 100 hospital discharge summaries (macro-averaged F1 0.94; dual-annotator agreement κ=0.87) provides complementary evidence of clinical-text utility. A safety mitigation pilot shows that context-disambiguation preprocessing reduces the highest-severity abbreviation-ambiguity error rate from 30% to 10% on a 30-case held-out set. These results show that parameter-efficient fine-tuning can deliver high-performance medical large language models while training only ∼4% of the model’s parameters and reducing memory use by roughly 75%, enabling development on low-cost consumer-grade hardware.

Zitieren

APA

El-Enen, M. A. A., Ismail, S. S., & Nazmy, T. M. (2026). Med-LLaMA3: Advancing Medical Question-Answering Through Parameter-Efficient Fine-Tuning of Large Language Models. Applied Sciences, 16(12), 6158. https://doi.org/10.3390/app16126158

Vancouver

El-Enen MAA, Ismail SS, Nazmy TM. Med-LLaMA3: Advancing Medical Question-Answering Through Parameter-Efficient Fine-Tuning of Large Language Models. Applied Sciences. 2026;16(12):6158. doi:10.3390/app16126158

BibTeX
@article{ElEnen2026,
  author = {El-Enen, Mohamed Ahmed Abo and Ismail, Sally S. and Nazmy, Taymoor Mohamed},
  title = {Med-LLaMA3: Advancing Medical Question-Answering Through Parameter-Efficient Fine-Tuning of Large Language Models},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {12},
  pages = {6158},
  doi = {10.3390/app16126158}
}
Kostenlos starten →