EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Text Psychometrics: Assessing psychological constructs in text using natural language processing

Daniel M Low iD, Patrick Mair, Matthew K Nock, Satrajit S Ghosh

PLOS Digital Health · 12.11.2026 · 5 (11) · e0001757

DOI: 10.1371/journal.pdig.0001757 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Digital Health, online erschienen am 12.11.2026. Verfasst von Daniel M Low und 3 weiteren Autor:innen. Beteiligte Einrichtung: Child Mind Institute, New York, New York, United States of America. Die Arbeit nennt 130 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NIDCD NIH HHS, NIMH NIH HHS, National Institute of Mental Health.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have revolutionized natural language processing (NLP). Yet when used to assess psychological constructs in text, they are generally not evaluated for the types of validity, reliability, and standardization typically expected from traditional questionnaires with rating scales. This study bridges that gap by proposing a framework for evaluating the psychometric properties of NLP-based psychological assessments (including LLMs-based ones), an area of research we refer to as Text Psychometrics. We first review different NLP methods, compare their ability to address key challenges in psychological research such as explainability, and outline ways of evaluating them on many desirable psychometric properties. Our goal is to describe most psychometric properties that may be of interest, especially for psychology and NLP researchers who are less familiar with psychometrics or psychometricians who are less familiar with NLP. We then illustrate the tradeoffs of different NLP methods through two empirical studies. Study 1 classifies thousands of crisis counseling conversations and Reddit posts into different types of mental health issues and introduces a novel method to evaluate text models for content validity—the extent to which a test captures the full range of expressions of a single construct. Study 2 examines prospective criterion validity by estimating how 49 known suicide risk factors predict imminent risk in crisis counseling conversations. We find that mentioning lethal means for suicide, the police, depressed mood, and anxiety in text as well as the texter writing more than the counselor characterize the conversations that end up needing an emergency service intervention for imminent suicide risk, which showcases the interpretability of the method that was used. In sum, NLP studies in psychology often rely on only a few validation metrics; here, we demonstrate the need for broader psychometric evaluation and, as a step towards Text Psychometrics, provide practical examples and future directions that can help researchers evaluate these models.

Autor:innen und Institutionen

  • Child Mind Institute, New York, New York, United States of America.
  • Department of Psychology, Harvard University, Cambridge, Massachusetts, United States of America.
  • Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, United States of America.
  • Franciscan Children's Hospital, Boston, Massachusetts, United States of America.
  • Department of Epidemiology, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, United States of America.
  • McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America.

ORCID-Profile: Daniel M Low

Förderung

  • NIDCD NIH HHS
  • NIMH NIH HHS
  • National Institute of Mental Health
  • National Institute on Deafness and Other Communication Disorders
  • RallyPoint Fellowship

Zitieren

APA

Low, D. M., Mair, P., Nock, M. K., & Ghosh, S. S. (2026). Text Psychometrics: Assessing psychological constructs in text using natural language processing. PLOS Digital Health, 5(11), e0001757. https://doi.org/10.1371/journal.pdig.0001757

Vancouver

Low DM, Mair P, Nock MK, Ghosh SS. Text Psychometrics: Assessing psychological constructs in text using natural language processing. PLOS Digital Health. 2026;5(11):e0001757. doi:10.1371/journal.pdig.0001757

BibTeX
@article{Low2026,
  author = {Low, Daniel M and Mair, Patrick and Nock, Matthew K and Ghosh, Satrajit S},
  title = {Text Psychometrics: Assessing psychological constructs in text using natural language processing},
  journal = {PLOS Digital Health},
  year = {2026},
  volume = {5},
  number = {11},
  pages = {e0001757},
  doi = {10.1371/journal.pdig.0001757}
}
Kostenlos starten →