EUVIMEDEUVIMED entdecken
Open Access

Intervention in Health Misinformation Using Large Language Models for Automated Detection, Thematic Analysis, and Inoculation: Case Study on COVID-19

Samira Malek, Christopher Griffin, Robert D Fraleigh, Robert Lennon, Vishal Monga, Lijiang Shen

Journal of Medical Internet Research · 08.01.2026 · 28 · e75500-e75500

DOI: 10.2196/75500 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Medical Internet Research, online erschienen am 08.01.2026. Verfasst von Samira Malek und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science and Engineering, Pennsylvania State University, University Park, PA, United States. Hauptthemen laut MEDLINE: COVID-19, Social Media, Communication. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 71 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The rapid growth of social media as an information channel has enabled the swift spread of inaccurate or false health information, significantly impacting public health. This widespread dissemination of misinformation has caused confusion, eroded trust in health authorities, led to noncompliance with health guidelines, and encouraged risky health behaviors. Understanding the dynamics of misinformation on social media is essential for devising effective public health communication strategies.

Objective

This study aims to present a comprehensive and automated approach that leverages large language models (LLMs) and machine learning techniques to detect misinformation on social media, uncover the underlying causes and themes, and generate refutation arguments, facilitating control of its spread and promoting public health outcomes by inoculating people against health misinformation.

Methods

We use 2 datasets to train 3 LLMs, namely, BERT, T5, and GPT-2, to classify documents into 2 categories: misinformation and nonmisinformation. In addition, we use a separate dataset to identify misinformation topics. To analyze these topics, we applied 3 topic modeling algorithms—Latent Dirichlet Allocation, Top2Vec, and BERTopic—and selected the optimal model based on performance evaluated across 3 metrics. Using a prompting approach, we extract sentence-level representations for the topics to uncover their underlying themes. Finally, we design a prompt text capable of identifying misinformation themes effectively.

Results

The trained BERT model demonstrated exceptional performance, achieving 98% accuracy in classifying misinformation and nonmisinformation, with a 44% reduction in false-positive rates for artificial intelligence–generated misinformation. Among the 3 topic modeling approaches used, BERTopic outperformed the others, achieving the highest metrics with a Coherence Value of 0.41, Normalized Pointwise Mutual Information of −0.086, and Inverse Rank-Biased Overlap of 0.99. To address the issue of unclassified documents, we developed an algorithm to assign each document to its closest topic. In addition, we proposed a novel method using prompt engineering to generate sentence-level representations for each topic, achieving a 99.6% approval rate as “appropriate” or “somewhat appropriate” by 3 independent raters. We further designed a prompt text to identify themes of misinformation topics and developed another prompt capable of detecting misinformation themes with 82% accuracy.

Conclusions

This study presents a comprehensive and automated approach to addressing health misinformation on social media using advanced machine learning and natural language processing techniques. By leveraging LLMs and prompt engineering, the system effectively detects misinformation, identifies underlying themes, and provides explanatory responses to combat its spread. The proposed method was tested on an English language COVID-19–related dataset and has not been evaluated on real-world online social media data; the experiments were conducted offline.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

COVID-19Social MediaCommunicationLarge Language ModelsHumansMachine LearningPandemicsSARS-CoV-2

Stichwörter der Autor:innen

COVID-19large language modelsmachine learningmisinformationprompt engineeringtopic modeling

Autor:innen und Institutionen

  • Department of Computer Science and Engineering, Pennsylvania State University, University Park, PA, United States.
  • Applied Research Laboratory, Pennsylvania State University, University Park, PA, United States.
  • Department of Mathematics, Pennsylvania State University, University Park, PA, United States.
  • PrimeCare Medical, Harrisburg, PA, United States.
  • Department of Electrical Engineering, Pennsylvania State University, University Park, PA, United States.
  • Department of Communication Arts and Sciences, Pennsylvania State University, 211 Sparks Building, University Park, PA, 16802, United States, 1 (814) 865-1736.

Zitieren

APA

Malek, S., Griffin, C., Fraleigh, R. D., Lennon, R., Monga, V., & Shen, L. (2026). Intervention in Health Misinformation Using Large Language Models for Automated Detection, Thematic Analysis, and Inoculation: Case Study on COVID-19. Journal of Medical Internet Research, 28, e75500-e75500. https://doi.org/10.2196/75500

Vancouver

Malek S, Griffin C, Fraleigh RD, Lennon R, Monga V, Shen L. Intervention in Health Misinformation Using Large Language Models for Automated Detection, Thematic Analysis, and Inoculation: Case Study on COVID-19. Journal of Medical Internet Research. 2026;28:e75500-e75500. doi:10.2196/75500

BibTeX
@article{Malek2026,
  author = {Malek, Samira and Griffin, Christopher and Fraleigh, Robert D and Lennon, Robert and Monga, Vishal and Shen, Lijiang},
  title = {Intervention in Health Misinformation Using Large Language Models for Automated Detection, Thematic Analysis, and Inoculation: Case Study on COVID-19},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e75500--e75500},
  doi = {10.2196/75500}
}
Kostenlos starten →