Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI
Political Analysis · 2024-01 · 32 (1) · 84-100
DOI: 10.1017/pan.2023.20 ↗Kurz gefasst
Fachartikel in der Zeitschrift Political Analysis, online erschienen am 09.06.2023. Verfasst von Moritz Laurer und 3 weiteren Autor:innen. Die Arbeit wurde bisher 165-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Nederlandse Organisatie voor Wetenschappelijk Onderzoek, Heinrich Böll Stiftung.
Abstract (Originalsprache)
Supervised machine learning is an increasingly popular tool for analyzing large political text corpora. The main disadvantage of supervised machine learning is the need for thousands of manually annotated training data points. This issue is particularly important in the social sciences where most new research questions require new training data for a new task tailored to the specific research question. This paper analyses how deep transfer learning can help address this challenge by accumulating “prior knowledge” in language models. Models like BERT can learn statistical language patterns through pre-training (“language knowledge”), and reliance on task-specific data can be reduced by training on universal tasks like natural language inference (NLI; “task knowledge”). We demonstrate the benefits of transfer learning on a wide range of eight tasks. Across these eight tasks, our BERT-NLI model fine-tuned on 100 to 2,500 texts performs on average 10.7 to 18.3 percentage points better than classical models without transfer learning. Our study indicates that BERT-NLI fine-tuned on 500 texts achieves similar performance as classical models trained on around 5,000 texts. Moreover, we show that transfer learning works particularly well on imbalanced data. We conclude by discussing limitations of transfer learning and by outlining new opportunities for political science research.
Förderung
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- Heinrich Böll Stiftung
Zitieren
APA
Laurer, M., Atteveldt, W. V., Casas, A., & Welbers, K. (2024). Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI. Political Analysis, 32(1), 84-100. https://doi.org/10.1017/pan.2023.20
Vancouver
Laurer M, Atteveldt WV, Casas A, Welbers K. Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI. Political Analysis. 2024;32(1):84-100. doi:10.1017/pan.2023.20
BibTeX
@article{Laurer2024,
author = {Laurer, Moritz and Atteveldt, Wouter van and Casas, Andreu and Welbers, Kasper},
title = {Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI},
journal = {Political Analysis},
year = {2024},
volume = {32},
number = {1},
pages = {84--100},
doi = {10.1017/pan.2023.20}
}
EUVIMED