EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

AI and social science: Automatic classification tools for big data analysis in sociological research

Andrea Nucita iD, Assunta Penna, Antonia Cava, Giancarlo Iannizzotto, Massimo Mucciardi

PLOS One · 18.06.2026 · 21 (6) · e0351336

DOI: 10.1371/journal.pone.0351336 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 18.06.2026. Verfasst von Andrea Nucita und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Cognitive Sciences, Psychology, Education, and Cultural Studies, University of Messina, Messina, Italy. Hauptthemen laut MEDLINE: Big Data, Social Sciences, Social Media, Artificial Intelligence. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Università degli Studi di Messina.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study examines the use of Social Network Sites for public institutional communication through a sociological, data-driven lens, focusing on the challenges and potential of automated classification tools for data analysis. Although Large Language Models are increasingly used to process social media data, a key research gap remains: few studies systematically assess whether AI-based categorizations are as reliable as human coding, especially when categories are semantically ambiguous. The research addresses the following questions: How reliable are AI-generated classifications compared to those made by human experts? Is human–machine agreement comparable to the level of agreement observed among human coders? To experimentally test this approach, we conducted a case study on Facebook posts published by two Italian universities (March 2020–March 2023), classified into eight categories of public institutional communication. Three researchers independently annotated the dataset. Human annotations are used as a benchmark to assess agreement patterns and to compare them with classifications produced by AI-based systems. Results show substantial interpretive ambiguity across several categories, mirrored by variability among human coders. Nonetheless, automated models achieve agreement with human classifications that is broadly comparable to inter-coder agreement. Overall, the findings support integrating AI as an additional coder within hybrid workflows to enable scalable and transparent sociological analysis of complex social media data.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Big DataSocial SciencesSocial MediaArtificial IntelligenceSociologyHumansData AnalyticsLarge Language ModelsClassification Algorithms

Autor:innen und Institutionen

  • Department of Cognitive Sciences, Psychology, Education, and Cultural Studies, University of Messina, Messina, Italy.

ORCID-Profile: Andrea Nucita

Förderung

  • Università degli Studi di Messina

Zitieren

APA

Nucita, A., Penna, A., Cava, A., Iannizzotto, G., & Mucciardi, M. (2026). AI and social science: Automatic classification tools for big data analysis in sociological research. PLOS One, 21(6), e0351336. https://doi.org/10.1371/journal.pone.0351336

Vancouver

Nucita A, Penna A, Cava A, Iannizzotto G, Mucciardi M. AI and social science: Automatic classification tools for big data analysis in sociological research. PLOS One. 2026;21(6):e0351336. doi:10.1371/journal.pone.0351336

BibTeX
@article{Nucita2026,
  author = {Nucita, Andrea and Penna, Assunta and Cava, Antonia and Iannizzotto, Giancarlo and Mucciardi, Massimo},
  title = {AI and social science: Automatic classification tools for big data analysis in sociological research},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {6},
  pages = {e0351336},
  doi = {10.1371/journal.pone.0351336}
}
Kostenlos starten →