Social Desirability Bias in Health Screening Questionnaire Responses Generated by OpenAI Models: Quantitative Experimental Study
JMIR Formative Research · 03.11.2026 · 10 · e92057
DOI: 10.2196/92057 ↗Kurz gefasst
Fachartikel in der Zeitschrift JMIR Formative Research, online erschienen am 03.11.2026. Verfasst von Andrew O'Malley. Beteiligte Einrichtung: School of Medicine, University of St Andrews, St Andrews, Scotland, United Kingdom. Hauptthemen laut MEDLINE: Social Desirability, Mass Screening. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 26 Literaturangaben.
Abstract (Originalsprache)
Background
Social desirability bias, the tendency to respond in ways that present the respondent favorably, is well documented in human self-report and has recently been observed in large language models (LLMs) completing personality inventories. Whether it also affects LLM responses to validated health screening instruments, where LLMs are increasingly proposed as simulated patients or synthetic respondents, has not been examined.
Objective
This formative study aimed to determine whether 4 OpenAI models alter their responses to health screening questionnaires when the instrument and its evaluative purpose can be inferred from the prompt and to quantify the size and direction of any shift.
Methods
GPT-4o, GPT-4o mini, o1-mini, and o1 were prompted through the OpenAI application programming interface in April 2025 to complete the Generalized Anxiety Disorder-7 (GAD-7), Patient Health Questionnaire-9 (PHQ-9), Alcohol Use Disorders Identification Test (AUDIT), and FANTASTIC lifestyle assessment while adopting the persona of a random member of the public. Each instrument was administered under a naive condition, wherein every item was presented alone in a separate call so that the instrument could not be recognized, and an informed condition, in which all items were presented together, with 100 completed trials per model, instrument, and condition. Naive and informed scores were compared within each model using independent-samples 2-tailed t tests with Bonferroni correction for 16 comparisons, with effect sizes expressed as Cohen d and in units of published general-population SDs. Published human norms served only as descriptive reference points.
Results
In 14 of 16 model-instrument combinations, scores shifted significantly toward the socially desirable pole when the instrument was recognizable (all P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Medicine, University of St Andrews, St Andrews, Scotland, United Kingdom.
ORCID-Profile: Andrew O'Malley
Zitieren
APA
O'Malley, A. (2026). Social Desirability Bias in Health Screening Questionnaire Responses Generated by OpenAI Models: Quantitative Experimental Study. JMIR Formative Research, 10, e92057. https://doi.org/10.2196/92057
Vancouver
O'Malley A. Social Desirability Bias in Health Screening Questionnaire Responses Generated by OpenAI Models: Quantitative Experimental Study. JMIR Formative Research. 2026;10:e92057. doi:10.2196/92057
BibTeX
@article{OMalley2026,
author = {O'Malley, Andrew},
title = {Social Desirability Bias in Health Screening Questionnaire Responses Generated by OpenAI Models: Quantitative Experimental Study},
journal = {JMIR Formative Research},
year = {2026},
volume = {10},
pages = {e92057},
doi = {10.2196/92057}
}
EUVIMED