EUVIMEDEUVIMED entdecken
Crossref

StatLLaMA: Multi-stage training for domain-optimized statistical large language models

Jing-Yi Zeng, Guan-Hua Huang

Journal of Data Science, Statistics, and Visualisation · 09.06.2026 · 6 (4)

DOI: 10.52933/jdssv.v6i4.171 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Data Science, Statistics, and Visualisation, online erschienen am 09.06.2026. Verfasst von Jing-Yi Zeng und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study investigates how to efficiently build a domain-specialized large language model (LLM) for statistics using the lightweight LLaMA-3.2-3B family as the foundation model (FM). We systematically compare three multi-stage training pipelines—starting from a base FM with no instruction-following capability, a base FM augmented with post-hoc instruction tuning, and an instruction-tuned FM with strong general reasoning abilities—across continual pretraining, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF) preference alignment. Results show that pipelines beginning with a base FM fail to develop meaningful statistical reasoning, even after extensive instruction tuning, SFT, or RLHF alignment. In contrast, starting from LLaMA-3.2-3B-Instruct enables effective domain specialization. A comprehensive evaluation of SFT variants reveals clear trade-offs between domain expertise and general reasoning ability. We further demonstrate that direct preference optimization provides stable and effective RLHF preference alignment. The final model, StatLLaMA, achieves strong and balanced performance on benchmarks of mathematical reasoning, common-sense reasoning, and statistical expertise, offering a practical blueprint for developing resource-efficient statistical LLMs. The code is available at https://github.com/HuangDLab/StatLLaMA.

Zitieren

APA

Zeng, J., & Huang, G. (2026). StatLLaMA: Multi-stage training for domain-optimized statistical large language models. Journal of Data Science, Statistics, and Visualisation, 6(4). https://doi.org/10.52933/jdssv.v6i4.171

Vancouver

Zeng J, Huang G. StatLLaMA: Multi-stage training for domain-optimized statistical large language models. Journal of Data Science, Statistics, and Visualisation. 2026;6(4). doi:10.52933/jdssv.v6i4.171

BibTeX
@article{Zeng2026,
  author = {Zeng, Jing-Yi and Huang, Guan-Hua},
  title = {StatLLaMA: Multi-stage training for domain-optimized statistical large language models},
  journal = {Journal of Data Science, Statistics, and Visualisation},
  year = {2026},
  volume = {6},
  number = {4},
  doi = {10.52933/jdssv.v6i4.171}
}
Kostenlos starten →