EUVIMEDObjevte EUVIMED
VergleichsstudieOpen access

Analyzing Nurse Sentiment in a Dutch Regional Newspaper During the COVID-19 Pandemic: Comparative Content Analysis of GPT-5, Gemini 2.5 Pro, and Human Coding

Bram van den Berkmortel, Daan Westra, Rachel Gifford, Frank Christian van de Baan

Journal of Medical Internet Research · 2026-08-13 · 28 · e98374-e98374

DOI: 10.2196/98374 ↗

At a glance

Vergleichsstudie in Journal of Medical Internet Research, published online on 2026-08-13. Written by Bram van den Berkmortel and 3 co-authors. Institution: Department of Health Services Research, Care and Public Health Research Institute, Maastricht University, Duboisdomein 30, Maastricht, Limburg, 6229G…. Major MEDLINE topics: COVID-19, Nurses, Newspapers as Topic. 20 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Large language models (LLMs) are increasingly used in qualitative research, but their reliability compared to human analysis, especially on large, non-English datasets, is unclear. Previous studies on older models (like GPT-4) show limitations in nuance and token capacity.

Objective

This study compared the qualitative analysis capabilities of OpenAI’s GPT-5 and Google’s Gemini 2.5 Pro (Gemini) with a human qualitative analysis. The study uses a large dataset of 317 Dutch newspaper articles from January 1, 2020, to December 31, 2023, investigating the sentiment toward nurses during the COVID-19 pandemic.

Methods

The study used a 2-phase methodology. First, a thematic comparison was conducted where the human researchers, GPT-5, and Gemini independently generated inductive coding trees from the entire corpus. Second, a comparative test was performed where all 3 coders applied a predefined codebook to a 10% stratified random subsample. The human baseline was validated through double-coding by 2 independent researchers, achieving an acceptable intercoder reliability (α=0.703). The AI analysis was iterative, using model-optimized prompts and an article-by-article approach.

Results

Both AI models successfully identified third-order themes (eg, “Health care heroes”) consistent with the data. In deductive application, however, both models systematically overcoded compared to the human consensus (181 and 183 codes vs 138), resulting in low intercoder reliability against the human baseline (α=0.487) for GPT-5 and (α=0.507) for Gemini.

Conclusions

This study suggests a potential divergence in analytical logic. The observed coding frequencies indicate that LLMs may default to semantic presence (literal frequency), whereas human coders appear to prioritize interpretive significance (contextual weight), leading to systematic overcoding. Consequently, this article argues that LLMs should not be viewed as autonomous researchers but as high-sensitivity filtering instruments requiring human calibration. This study concludes that AI can serve as a valuable assistant for qualitative researchers. Still, it benefits from a rigorous, iterative, and human-in-the-loop approach to manage methodological friction and ensure nuanced, valid analysis.

Subjects

MeSH terms (MEDLINE)

COVID-19NursesNewspapers as TopicHumansNetherlandsLarge Language ModelsPandemicsSARS-CoV-2Generative Artificial IntelligenceQualitative Research

Author keywords

AICOVID-19comparative studyhealth services researchlarge language modelsmedical informaticsnatural language processingnursingqualitative researchsentiment analysis

Authors and institutions

  • Department of Health Services Research, Care and Public Health Research Institute, Maastricht University, Duboisdomein 30, Maastricht, Limburg, 6229GT, The Netherlands, 31 043 38 81 553.

Cite

APA

van den Berkmortel, B., Westra, D., Gifford, R., & van de Baan, F. C. (2026). Analyzing Nurse Sentiment in a Dutch Regional Newspaper During the COVID-19 Pandemic: Comparative Content Analysis of GPT-5, Gemini 2.5 Pro, and Human Coding. Journal of Medical Internet Research, 28, e98374-e98374. https://doi.org/10.2196/98374

Vancouver

van den Berkmortel B, Westra D, Gifford R, van de Baan FC. Analyzing Nurse Sentiment in a Dutch Regional Newspaper During the COVID-19 Pandemic: Comparative Content Analysis of GPT-5, Gemini 2.5 Pro, and Human Coding. Journal of Medical Internet Research. 2026;28:e98374-e98374. doi:10.2196/98374

BibTeX
@article{vandenBerkmortel2026,
  author = {van den Berkmortel, Bram and Westra, Daan and Gifford, Rachel and van de Baan, Frank Christian},
  title = {Analyzing Nurse Sentiment in a Dutch Regional Newspaper During the COVID-19 Pandemic: Comparative Content Analysis of GPT-5, Gemini 2.5 Pro, and Human Coding},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e98374--e98374},
  doi = {10.2196/98374}
}
Start for free →