EUVIMEDOpdag EUVIMED
Crossref

Social Desirability Bias in Health Screening Questionnaire Responses Generated by OpenAI Models: Quantitative Experimental Study

Andrew O'Malley iD

JMIR Formative Research · 2026-11-03 · 10 · e92057

DOI: 10.2196/92057 ↗

At a glance

Journal article in JMIR Formative Research, published online on 2026-11-03. Written by Andrew O'Malley. Institution: School of Medicine, University of St Andrews, St Andrews, Scotland, United Kingdom. Major MEDLINE topics: Social Desirability, Mass Screening. Cited 1 times so far (Crossref); 26 references.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Social desirability bias, the tendency to respond in ways that present the respondent favorably, is well documented in human self-report and has recently been observed in large language models (LLMs) completing personality inventories. Whether it also affects LLM responses to validated health screening instruments, where LLMs are increasingly proposed as simulated patients or synthetic respondents, has not been examined.

Objective

This formative study aimed to determine whether 4 OpenAI models alter their responses to health screening questionnaires when the instrument and its evaluative purpose can be inferred from the prompt and to quantify the size and direction of any shift.

Methods

GPT-4o, GPT-4o mini, o1-mini, and o1 were prompted through the OpenAI application programming interface in April 2025 to complete the Generalized Anxiety Disorder-7 (GAD-7), Patient Health Questionnaire-9 (PHQ-9), Alcohol Use Disorders Identification Test (AUDIT), and FANTASTIC lifestyle assessment while adopting the persona of a random member of the public. Each instrument was administered under a naive condition, wherein every item was presented alone in a separate call so that the instrument could not be recognized, and an informed condition, in which all items were presented together, with 100 completed trials per model, instrument, and condition. Naive and informed scores were compared within each model using independent-samples 2-tailed t tests with Bonferroni correction for 16 comparisons, with effect sizes expressed as Cohen d and in units of published general-population SDs. Published human norms served only as descriptive reference points.

Results

In 14 of 16 model-instrument combinations, scores shifted significantly toward the socially desirable pole when the instrument was recognizable (all P

Subjects

MeSH terms (MEDLINE)

Social DesirabilityMass ScreeningHumansLarge Language ModelsSurveys and QuestionnairesFemaleMaleAdultBiasMiddle Aged

Author keywords

generative AI in health carehealth screening questionnaireslarge language modelsmedical educationsocial desirability bias

Authors and institutions

  • School of Medicine, University of St Andrews, St Andrews, Scotland, United Kingdom.

ORCID profiles: Andrew O'Malley

Cite

APA

O'Malley, A. (2026). Social Desirability Bias in Health Screening Questionnaire Responses Generated by OpenAI Models: Quantitative Experimental Study. JMIR Formative Research, 10, e92057. https://doi.org/10.2196/92057

Vancouver

O'Malley A. Social Desirability Bias in Health Screening Questionnaire Responses Generated by OpenAI Models: Quantitative Experimental Study. JMIR Formative Research. 2026;10:e92057. doi:10.2196/92057

BibTeX
@article{OMalley2026,
  author = {O'Malley, Andrew},
  title = {Social Desirability Bias in Health Screening Questionnaire Responses Generated by OpenAI Models: Quantitative Experimental Study},
  journal = {JMIR Formative Research},
  year = {2026},
  volume = {10},
  pages = {e92057},
  doi = {10.2196/92057}
}
Start for free →