Cross-lingual Natural Language Processing on Limited Annotated Case/Radiology Reports in English and Japanese: Insights from the Real-MedNLP Workshop
Methods of Information in Medicine · 01.12.2024 · 63 (05/06) · 145-163
DOI: 10.1055/a-2405-2489 ↗Kurz gefasst
Fachartikel in der Zeitschrift Methods of Information in Medicine, erschienen am 01.12.2024. Verfasst von Shuntaro Yada und 3 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Science and Technology, Nara Institute of Science and Technology, Nara, Japan. Hauptthemen laut MEDLINE: Natural Language Processing, Multilingualism, Radiology. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: JST AIP Trilateral AI, JST AIP- MHLW Program, Ministry of Health, Labour and Welfare.
Abstract (Originalsprache)
Background
Textual datasets (corpora) are crucial for the application of natural language processing (NLP) models. However, corpus creation in the medical field is challenging, primarily because of privacy issues with raw clinical data such as health records. Thus, the existing clinical corpora are generally small and scarce. Medical NLP (MedNLP) methodologies perform well with limited data availability.
Objectives
We present the outcomes of the Real-MedNLP workshop, which was conducted using limited and parallel medical corpora. Real-MedNLP exhibits three distinct characteristics: (1) limited annotated documents: the training data comprise only a small set (∼100) of case reports (CRs) and radiology reports (RRs) that have been annotated. (2) Bilingually parallel: the constructed corpora are parallel in Japanese and English. (3) Practical tasks: the workshop addresses fundamental tasks, such as named entity recognition (NER) and applied practical tasks.
Methods
We propose three tasks: NER of ∼100 available documents (Task 1), NER based only on annotation guidelines for humans (Task 2), and clinical applications (Task 3) consisting of adverse drug effect (ADE) detection for CRs and identical case identification (CI) for RRs.
Results
Nine teams participated in this study. The best systems achieved 0.65 and 0.89 F1-scores for CRs and RRs in Task 1, whereas the top scores in Task 2 decreased by 50 to 70%. In Task 3, ADE reports were detected by up to 0.64 F1-score, and CI scored up to 0.96 binary accuracy.
Conclusion
Most systems adopt medical-domain–specific pretrained language models using data augmentation methods. Despite the challenge of limited corpus size in Tasks 1 and 2, recent approaches are promising because the partial match scores reached ∼0.8–0.9 F1-scores. Task 3 applications revealed that the different availabilities of external language resources affected the performance per language.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Graduate School of Science and Technology, Nara Institute of Science and Technology, Nara, Japan.
- 22nd Century Medical and Research Center, The University of Tokyo Hospital, Tokyo, Japan.
Förderung
- JST AIP Trilateral AI
- JST AIP- MHLW Program
- Ministry of Health, Labour and Welfare
- Japan Science and Technology Agency
Zitieren
APA
Yada, S., Nakamura, Y., Wakamiya, S., & Aramaki, E. (2024). Cross-lingual Natural Language Processing on Limited Annotated Case/Radiology Reports in English and Japanese: Insights from the Real-MedNLP Workshop. Methods of Information in Medicine, 63(05/06), 145-163. https://doi.org/10.1055/a-2405-2489
Vancouver
Yada S, Nakamura Y, Wakamiya S, Aramaki E. Cross-lingual Natural Language Processing on Limited Annotated Case/Radiology Reports in English and Japanese: Insights from the Real-MedNLP Workshop. Methods of Information in Medicine. 2024;63(05/06):145-163. doi:10.1055/a-2405-2489
BibTeX
@article{Yada2024,
author = {Yada, Shuntaro and Nakamura, Yuta and Wakamiya, Shoko and Aramaki, Eiji},
title = {Cross-lingual Natural Language Processing on Limited Annotated Case/Radiology Reports in English and Japanese: Insights from the Real-MedNLP Workshop},
journal = {Methods of Information in Medicine},
year = {2024},
volume = {63},
number = {05/06},
pages = {145--163},
doi = {10.1055/a-2405-2489}
}
EUVIMED