Natural Language Response Formats for Assessing Depression and Worry With Large Language Models: A Sequential Evaluation With Model Pre-Registration
Assessment · 2026-09 · 33 (6) · 927-953
DOI: 10.1177/10731911251364022 ↗Kurz gefasst
Fachartikel in der Zeitschrift Assessment, online erschienen am 20.09.2025. Verfasst von Zhuojun Gu und 3 weiteren Autor:innen. Beteiligte Einrichtung: Lund University, Skåne, Sweden. Hauptthemen laut MEDLINE: Depression, Anxiety, Natural Language Processing. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 87 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NIAAA NIH HHS, NIOSH CDC HHS, US CDC/NIOSH.
Themen: Depression
Abstract (Originalsprache)
Large language models can transform individuals’ mental health descriptions into scores that correlate with rating scales approaching theoretical upper limits. However, such analyses have combined word- and text responses with little known about their differences. We develop response formats ranging from closed-ended to open-ended: (a) select words from lists, write (b) descriptive words, (c) phrases, or (d) texts. Participants answered questions about their depression/worry using the response formats and related rating scales. Language responses were transformed into word embeddings and trained to rating scales. We compare the validity (concurrent, incremental, face, discriminant, and external validity) and reliability (prospective sample and test–retest reliability) of the response formats. Using the Sequential Evaluation with Model Pre-Registration design, machine-learning models were trained on a development dataset ( N = 963), and then pre-registered before tested on a prospective sample ( N = 145). The pre-registered models demonstrate strong validity and reliability, yielding high accuracy in the prospective sample ( r = .60–.79). Additionally, the models demonstrated external validity to self-reported sick-leave/healthcare visits, where the text-format yielded the strongest correlations (being higher/equal to rating scales for 9 of 12 cases). The overall high validity and reliability across formats suggest the possibility of choosing formats according to clinical needs.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Lund University, Skåne, Sweden.
- Stony Brook University, USA.
ORCID-Profile: Zhuojun Gu, H Andrew Schwartz
Förderung
- NIAAA NIH HHS
- NIOSH CDC HHS
- US CDC/NIOSH
- NIH-NIAAA
- eSSENCE@LU
- Forté Foundation
- Marcus och Amalia Wallenbergs minnesfond
- Vetenskapsrådet
Zitieren
APA
Gu, Z., Kjell, K., Schwartz, H. A., & Kjell, O. (2026). Natural Language Response Formats for Assessing Depression and Worry With Large Language Models: A Sequential Evaluation With Model Pre-Registration. Assessment, 33(6), 927-953. https://doi.org/10.1177/10731911251364022
Vancouver
Gu Z, Kjell K, Schwartz HA, Kjell O. Natural Language Response Formats for Assessing Depression and Worry With Large Language Models: A Sequential Evaluation With Model Pre-Registration. Assessment. 2026;33(6):927-953. doi:10.1177/10731911251364022
BibTeX
@article{Gu2026,
author = {Gu, Zhuojun and Kjell, Katarina and Schwartz, H Andrew and Kjell, Oscar},
title = {Natural Language Response Formats for Assessing Depression and Worry With Large Language Models: A Sequential Evaluation With Model Pre-Registration},
journal = {Assessment},
year = {2026},
volume = {33},
number = {6},
pages = {927--953},
doi = {10.1177/10731911251364022}
}
EUVIMED