EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

The Accuracy of Large Language Models With in‐Context Learning: How to Detect Alcohol More Accurately in Social Media Data

Jingda Du, Zhen He, Emmanuel Kuntsche, Dan Anderson‐Luxford, Jack Delmenico, Joshua Millward, Benjamin Riordan

Drug and Alcohol Review · 2026-11 · 45 (7)

DOI: 10.1111/dar.70286 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Drug and Alcohol Review, online erschienen am 19.11.2026. Verfasst von Jingda Du und 6 weiteren Autor:innen. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Australian Research Council, VicHealth.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Research suggests that alcohol‐related content is common on social media, but it is difficult to determine just how common it is with current methods (i.e., content analyses). Using large language models (LLM) without further training (known as zero‐shot learning (ZSL)) may be an efficient strategy for analysing large text data. We aimed to determine whether LLMs could accurately detect whether alcohol is mentioned in Instagram posts. Additionally, we aimed to identify how to improve results by: (i) comparing different models that can be used for ZSL; (ii) including examples in the prompt (in‐context learning); and (iii) breaking the posts into smaller text chunks prior to analysis.

Methods

We annotated whether alcohol was present or not in 8522 Instagram posts from top influencers. We then compared the predictions of different LLMs and techniques to the annotated data to estimate accuracy.

Results

We found variations in performance across models and strategies; however, overall, we observed that using recent LLMs for ZSL was accurate in detecting alcohol references in Instagram posts (72%–95% accuracy). Accuracy improved slightly when including examples in the prompt (F1 scores with examples 0.92 vs. without examples 0.90) and when splitting posts into smaller text chunks prior to analysis (F1 score for 45‐word chunks 0.92 vs. no splitting 0.90). Discussion and Conclusions LLMs can accurately estimate whether alcohol is referenced in text, particularly when including representative examples in a prompt (in‐context learning). The application of these approaches should allow for more accurate estimates of alcohol in social media posts.

Förderung

  • Australian Research Council
  • VicHealth

Zitieren

APA

Du, J., He, Z., Kuntsche, E., Anderson‐Luxford, D., Delmenico, J., Millward, J., & Riordan, B. (2026). The Accuracy of Large Language Models With in‐Context Learning: How to Detect Alcohol More Accurately in Social Media Data. Drug and Alcohol Review, 45(7). https://doi.org/10.1111/dar.70286

Vancouver

Du J, He Z, Kuntsche E, Anderson‐Luxford D, Delmenico J, Millward J, et al. The Accuracy of Large Language Models With in‐Context Learning: How to Detect Alcohol More Accurately in Social Media Data. Drug and Alcohol Review. 2026;45(7). doi:10.1111/dar.70286

BibTeX
@article{Du2026,
  author = {Du, Jingda and He, Zhen and Kuntsche, Emmanuel and Anderson‐Luxford, Dan and Delmenico, Jack and Millward, Joshua and Riordan, Benjamin},
  title = {The Accuracy of Large Language Models With in‐Context Learning: How to Detect Alcohol More Accurately in Social Media Data},
  journal = {Drug and Alcohol Review},
  year = {2026},
  volume = {45},
  number = {7},
  doi = {10.1111/dar.70286}
}
Kostenlos starten →