EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Neural machine translation of clinical text: an empirical investigation into multilingual pre-trained language models and transfer-learning

Lifeng Han, Serge Gladkoff, Gleb Erofeev, Irina Sorokina, Betty Galiano, Goran Nenadic

Frontiers in Digital Health · 26.02.2024 · 6

DOI: 10.3389/fdgth.2024.1211564 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Digital Health, online erschienen am 26.02.2024. Verfasst von Lifeng Han und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, The University of Manchester, Manchester, United Kingom. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Nuffield Foundation, UKRI/EPSRC.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Clinical text and documents contain very rich information and knowledge in healthcare, and their processing using state-of-the-art language technology becomes very important for building intelligent systems for supporting healthcare and social good. This processing includes creating language understanding models and translating resources into other natural languages to share domain-specific cross-lingual knowledge. In this work, we conduct investigations on clinical text machine translation by examining multilingual neural network models using deep learning such as Transformer based structures. Furthermore, to address the language resource imbalance issue, we also carry out experiments using a transfer learning methodology based on massive multilingual pre-trained language models (MMPLMs). The experimental results on three sub-tasks including (1) clinical case (CC), (2) clinical terminology (CT), and (3) ontological concept (OC) show that our models achieved top-level performances in the ClinSpEn-2022 shared task on English-Spanish clinical domain data. Furthermore, our expert-based human evaluations demonstrate that the small-sized pre-trained language model (PLM) outperformed the other two extra-large language models by a large margin in the clinical domain fine-tuning, which finding was never reported in the field. Finally, the transfer learning method works well in our experimental setting using the WMT21fb model to accommodate a new language space Spanish that was not seen at the pre-training stage within WMT21fb itself, which deserves more exploitation for clinical knowledge transformation, e.g. to investigate into more languages. These research findings can shed some light on domain-specific machine translation development, especially in clinical and healthcare fields. Further research projects can be carried out based on our work to improve healthcare text analytics and knowledge transformation. Our data is openly available for research purposes at: https://github.com/HECTA-UoM/ClinicalNMT .

Schlagwörter

Stichwörter der Autor:innen

Neural machine translationSpanish-English translationclinical knowledge transformationclinical text translationlarge language modelmultilingual pre-trained language modeltransfer learning

Autor:innen und Institutionen

  • Department of Computer Science, The University of Manchester, Manchester, United Kingom.
  • AI Lab, Logrus Global, Translation & Localization, Philadelphia, PA, United States.
  • Management Department, Ocean Translations, Rosario, Argentina.

Förderung

  • Nuffield Foundation
  • UKRI/EPSRC

Zitieren

APA

Han, L., Gladkoff, S., Erofeev, G., Sorokina, I., Galiano, B., & Nenadic, G. (2024). Neural machine translation of clinical text: an empirical investigation into multilingual pre-trained language models and transfer-learning. Frontiers in Digital Health, 6. https://doi.org/10.3389/fdgth.2024.1211564

Vancouver

Han L, Gladkoff S, Erofeev G, Sorokina I, Galiano B, Nenadic G. Neural machine translation of clinical text: an empirical investigation into multilingual pre-trained language models and transfer-learning. Frontiers in Digital Health. 2024;6. doi:10.3389/fdgth.2024.1211564

BibTeX
@article{Han2024,
  author = {Han, Lifeng and Gladkoff, Serge and Erofeev, Gleb and Sorokina, Irina and Galiano, Betty and Nenadic, Goran},
  title = {Neural machine translation of clinical text: an empirical investigation into multilingual pre-trained language models and transfer-learning},
  journal = {Frontiers in Digital Health},
  year = {2024},
  volume = {6},
  doi = {10.3389/fdgth.2024.1211564}
}
Kostenlos starten →