Textual overlap rather than domain alignment: A comparative study of fine-tuning strategies for specialised machine translation with large language models
PLOS One · 20.07.2026 · 21 (7) · e0352256
DOI: 10.1371/journal.pone.0352256 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift PLOS One, online erschienen am 20.07.2026. Verfasst von Lixue Yang und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Foreign Languages, Tianjin University of Technology and Education, Tianjin, China. Hauptthemen laut MEDLINE: Translating. Die Arbeit nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Tianjin Higher Education Postgraduate Education Reform Research Project.
Abstract (Originalsprache)
General-purpose large language models (LLMs) may struggle in specialised machine translation, but the conditions under which fine-tuning improves translation performance remain unclear. This study compares full-parameter fine-tuning (FPFT) and parameter-efficient fine-tuning (PEFT) for Chinese-English political discourse translation using a purpose-built corpus and the Qwen3-14B model. Translation performance was assessed on three 50-item test sets using BLEU, ROUGE-L F1, METEOR, and BERTScore F1, together with BLEU pass-rate likelihood-ratio G 2 tests, paired t-tests, and paired Cohen’s d z for item-level score differences. The results reveal a clear contrast between unseen in-domain evaluation, maximum-overlap benchmarking, and semantically related but non-fine-tuned evaluation. On Test Set A and Test Set C, neither fine-tuning strategy produced a statistically significant BLEU pass-rate advantage over the base model, and paired tests across the continuous metrics did not show consistent fine-tuning gains. On Test Set B, which was sampled from the fine-tuning corpus, both fine-tuned models substantially outperformed the base model across all four metrics, with FPFT achieving the highest scores and PEFT providing a more computationally efficient alternative. These findings indicate that textual overlap between training and deployment data, rather than broad domain similarity alone, strongly conditions the observed benefit of fine-tuning. The study offers an empirically grounded framework for selecting fine-tuning strategies in specialised machine translation.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- School of Foreign Languages, Tianjin University of Technology and Education, Tianjin, China.
ORCID-Profile: Lixue Yang
Förderung
- Tianjin Higher Education Postgraduate Education Reform Research Project
Zitieren
APA
Yang, L., Zhu, J., & Zhang, Z. (2026). Textual overlap rather than domain alignment: A comparative study of fine-tuning strategies for specialised machine translation with large language models. PLOS One, 21(7), e0352256. https://doi.org/10.1371/journal.pone.0352256
Vancouver
Yang L, Zhu J, Zhang Z. Textual overlap rather than domain alignment: A comparative study of fine-tuning strategies for specialised machine translation with large language models. PLOS One. 2026;21(7):e0352256. doi:10.1371/journal.pone.0352256
BibTeX
@article{Yang2026,
author = {Yang, Lixue and Zhu, Jiaxin and Zhang, Zengxin},
title = {Textual overlap rather than domain alignment: A comparative study of fine-tuning strategies for specialised machine translation with large language models},
journal = {PLOS One},
year = {2026},
volume = {21},
number = {7},
pages = {e0352256},
doi = {10.1371/journal.pone.0352256}
}
EUVIMED