EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI

Moritz Laurer, Wouter van Atteveldt, Andreu Casas, Kasper Welbers

Political Analysis · 2024-01 · 32 (1) · 84-100

DOI: 10.1017/pan.2023.20 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Political Analysis, online erschienen am 09.06.2023. Verfasst von Moritz Laurer und 3 weiteren Autor:innen. Die Arbeit wurde bisher 165-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Nederlandse Organisatie voor Wetenschappelijk Onderzoek, Heinrich Böll Stiftung.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Supervised machine learning is an increasingly popular tool for analyzing large political text corpora. The main disadvantage of supervised machine learning is the need for thousands of manually annotated training data points. This issue is particularly important in the social sciences where most new research questions require new training data for a new task tailored to the specific research question. This paper analyses how deep transfer learning can help address this challenge by accumulating “prior knowledge” in language models. Models like BERT can learn statistical language patterns through pre-training (“language knowledge”), and reliance on task-specific data can be reduced by training on universal tasks like natural language inference (NLI; “task knowledge”). We demonstrate the benefits of transfer learning on a wide range of eight tasks. Across these eight tasks, our BERT-NLI model fine-tuned on 100 to 2,500 texts performs on average 10.7 to 18.3 percentage points better than classical models without transfer learning. Our study indicates that BERT-NLI fine-tuned on 500 texts achieves similar performance as classical models trained on around 5,000 texts. Moreover, we show that transfer learning works particularly well on imbalanced data. We conclude by discussing limitations of transfer learning and by outlining new opportunities for political science research.

Förderung

  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek
  • Heinrich Böll Stiftung

Zitieren

APA

Laurer, M., Atteveldt, W. V., Casas, A., & Welbers, K. (2024). Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI. Political Analysis, 32(1), 84-100. https://doi.org/10.1017/pan.2023.20

Vancouver

Laurer M, Atteveldt WV, Casas A, Welbers K. Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI. Political Analysis. 2024;32(1):84-100. doi:10.1017/pan.2023.20

BibTeX
@article{Laurer2024,
  author = {Laurer, Moritz and Atteveldt, Wouter van and Casas, Andreu and Welbers, Kasper},
  title = {Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI},
  journal = {Political Analysis},
  year = {2024},
  volume = {32},
  number = {1},
  pages = {84--100},
  doi = {10.1017/pan.2023.20}
}
Kostenlos starten →