EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Textual overlap rather than domain alignment: A comparative study of fine-tuning strategies for specialised machine translation with large language models

Lixue Yang iD, Jiaxin Zhu, Zengxin Zhang

PLOS One · 20.07.2026 · 21 (7) · e0352256

DOI: 10.1371/journal.pone.0352256 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift PLOS One, online erschienen am 20.07.2026. Verfasst von Lixue Yang und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Foreign Languages, Tianjin University of Technology and Education, Tianjin, China. Hauptthemen laut MEDLINE: Translating. Die Arbeit nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Tianjin Higher Education Postgraduate Education Reform Research Project.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

General-purpose large language models (LLMs) may struggle in specialised machine translation, but the conditions under which fine-tuning improves translation performance remain unclear. This study compares full-parameter fine-tuning (FPFT) and parameter-efficient fine-tuning (PEFT) for Chinese-English political discourse translation using a purpose-built corpus and the Qwen3-14B model. Translation performance was assessed on three 50-item test sets using BLEU, ROUGE-L F1, METEOR, and BERTScore F1, together with BLEU pass-rate likelihood-ratio G 2 tests, paired t-tests, and paired Cohen’s d z for item-level score differences. The results reveal a clear contrast between unseen in-domain evaluation, maximum-overlap benchmarking, and semantically related but non-fine-tuned evaluation. On Test Set A and Test Set C, neither fine-tuning strategy produced a statistically significant BLEU pass-rate advantage over the base model, and paired tests across the continuous metrics did not show consistent fine-tuning gains. On Test Set B, which was sampled from the fine-tuning corpus, both fine-tuned models substantially outperformed the base model across all four metrics, with FPFT achieving the highest scores and PEFT providing a more computationally efficient alternative. These findings indicate that textual overlap between training and deployment data, rather than broad domain similarity alone, strongly conditions the observed benefit of fine-tuning. The study offers an empirically grounded framework for selecting fine-tuning strategies in specialised machine translation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

TranslatingLarge Language ModelsHumansSemantics

Autor:innen und Institutionen

  • School of Foreign Languages, Tianjin University of Technology and Education, Tianjin, China.

ORCID-Profile: Lixue Yang

Förderung

  • Tianjin Higher Education Postgraduate Education Reform Research Project

Zitieren

APA

Yang, L., Zhu, J., & Zhang, Z. (2026). Textual overlap rather than domain alignment: A comparative study of fine-tuning strategies for specialised machine translation with large language models. PLOS One, 21(7), e0352256. https://doi.org/10.1371/journal.pone.0352256

Vancouver

Yang L, Zhu J, Zhang Z. Textual overlap rather than domain alignment: A comparative study of fine-tuning strategies for specialised machine translation with large language models. PLOS One. 2026;21(7):e0352256. doi:10.1371/journal.pone.0352256

BibTeX
@article{Yang2026,
  author = {Yang, Lixue and Zhu, Jiaxin and Zhang, Zengxin},
  title = {Textual overlap rather than domain alignment: A comparative study of fine-tuning strategies for specialised machine translation with large language models},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {7},
  pages = {e0352256},
  doi = {10.1371/journal.pone.0352256}
}
Kostenlos starten →