EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Natural Language Response Formats for Assessing Depression and Worry With Large Language Models: A Sequential Evaluation With Model Pre-Registration

Zhuojun Gu iD, Katarina Kjell, H Andrew Schwartz iD, Oscar Kjell

Assessment · 2026-09 · 33 (6) · 927-953

DOI: 10.1177/10731911251364022 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Assessment, online erschienen am 20.09.2025. Verfasst von Zhuojun Gu und 3 weiteren Autor:innen. Beteiligte Einrichtung: Lund University, Skåne, Sweden. Hauptthemen laut MEDLINE: Depression, Anxiety, Natural Language Processing. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 87 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NIAAA NIH HHS, NIOSH CDC HHS, US CDC/NIOSH.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models can transform individuals’ mental health descriptions into scores that correlate with rating scales approaching theoretical upper limits. However, such analyses have combined word- and text responses with little known about their differences. We develop response formats ranging from closed-ended to open-ended: (a) select words from lists, write (b) descriptive words, (c) phrases, or (d) texts. Participants answered questions about their depression/worry using the response formats and related rating scales. Language responses were transformed into word embeddings and trained to rating scales. We compare the validity (concurrent, incremental, face, discriminant, and external validity) and reliability (prospective sample and test–retest reliability) of the response formats. Using the Sequential Evaluation with Model Pre-Registration design, machine-learning models were trained on a development dataset ( N = 963), and then pre-registered before tested on a prospective sample ( N = 145). The pre-registered models demonstrate strong validity and reliability, yielding high accuracy in the prospective sample ( r = .60–.79). Additionally, the models demonstrated external validity to self-reported sick-leave/healthcare visits, where the text-format yielded the strongest correlations (being higher/equal to rating scales for 9 of 12 cases). The overall high validity and reliability across formats suggest the possibility of choosing formats according to clinical needs.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

DepressionAnxietyNatural Language ProcessingHumansLarge Language ModelsReproducibility of ResultsFemaleAdultMaleProspective StudiesMiddle AgedMachine LearningYoung Adult

Stichwörter der Autor:innen

anxietyartificial intelligencedepressionlarge language modelsnatural languagenatural language processingpsychological assessment

Autor:innen und Institutionen

  • Lund University, Skåne, Sweden.
  • Stony Brook University, USA.

ORCID-Profile: Zhuojun Gu, H Andrew Schwartz

Förderung

  • NIAAA NIH HHS
  • NIOSH CDC HHS
  • US CDC/NIOSH
  • NIH-NIAAA
  • eSSENCE@LU
  • Forté Foundation
  • Marcus och Amalia Wallenbergs minnesfond
  • Vetenskapsrådet

Zitieren

APA

Gu, Z., Kjell, K., Schwartz, H. A., & Kjell, O. (2026). Natural Language Response Formats for Assessing Depression and Worry With Large Language Models: A Sequential Evaluation With Model Pre-Registration. Assessment, 33(6), 927-953. https://doi.org/10.1177/10731911251364022

Vancouver

Gu Z, Kjell K, Schwartz HA, Kjell O. Natural Language Response Formats for Assessing Depression and Worry With Large Language Models: A Sequential Evaluation With Model Pre-Registration. Assessment. 2026;33(6):927-953. doi:10.1177/10731911251364022

BibTeX
@article{Gu2026,
  author = {Gu, Zhuojun and Kjell, Katarina and Schwartz, H Andrew and Kjell, Oscar},
  title = {Natural Language Response Formats for Assessing Depression and Worry With Large Language Models: A Sequential Evaluation With Model Pre-Registration},
  journal = {Assessment},
  year = {2026},
  volume = {33},
  number = {6},
  pages = {927--953},
  doi = {10.1177/10731911251364022}
}
Kostenlos starten →