EUVIMEDScopri EUVIMED
VergleichsstudieOpen access

Personality Inference From Semistructured Interviews Using Large Language Models: Comparative Evaluation Study

Jianfeng Zhu, Julina Maharjan, Xinyu Li, Karin G Coifman, Ruoming Jin

Journal of Medical Internet Research · 2026-10-16 · 28 · e87129-e87129

DOI: 10.2196/87129 ↗

At a glance

Vergleichsstudie in Journal of Medical Internet Research, published online on 2026-10-16. Written by Jianfeng Zhu and 4 co-authors. Institution: Department of Computer Science, Kent State University, 241 Mathematical Sciences Building, Kent, OH, 44242, United States, 1 3306729107. Major MEDLINE topics: Large Language Models, Personality, Interviews as Topic. 83 references. The full text is freely available in PubMed Central. Funded by: NIMH NIH HHS.

AbstractSubjectsDati bibliograficiCiteRelated studies

Abstract (original language)

Background

Personality traits are central to understanding human behavior and life outcomes, yet their assessment remains challenging due to biases and limited expressiveness in traditional self-report inventories. The emergence of large language models (LLMs) creates new opportunities to infer personality from naturalistic text, but it remains unclear whether these systems can reliably predict validated personality traits or whether their predictions primarily reflect superficial linguistic patterns.

Objective

This study evaluates the ability of LLM-based methods to predict continuous Big Five personality scores (1‐5 Likert scale) from semistructured interview transcripts and compares the performance of different modeling paradigms.

Methods

We analyzed semistructured interview transcripts from 518 participants, each paired with validated self-reported Big Five scores measured using the 10-item Big Five Inventory (BFI-10). We compared four paradigms: (1) zero-shot and chain-of-thought prompting with GPT-5 mini; (2) parameter-efficient fine-tuning (low-rank adaptation) applied to open-source RoBERTa-base and LLaMA-2-7B models; (3) supervised fine-tuning of GPT-4.1 mini across two independent training runs; and (4) embedding-based regression using 4 pretrained encoders (all-mpnet-base-v2, E5-large-v2, text-embedding-3-small, and text-embedding-3-large) with a multioutput ridge regression.

Results

Supervised fine-tuning yielded the strongest overall performance among the evaluated approaches. In the best-performing run, correlations reached r =0.718 for conscientiousness and r =0.631 for agreeableness. Neuroticism showed moderate and consistent performance across the 2 fine-tuning runs, whereas extraversion was less stable across runs. Embedding-based regression, particularly with text-embedding-3-large, achieved competitive results for conscientiousness and agreeableness, although it remained below supervised fine-tuning. Prompt-based methods showed weak and inconsistent performance across traits. Openness was generally difficult to predict, although RoBERTa-base with embedding-based low-rank adaptation fine-tuning showed modest but statistically significant performance, indicating trait- and method-dependent variability.

Conclusions

LLM-based approaches exhibit trait-dependent performance in predicting personality from naturalistic dialogue, with fine-tuning substantially improving correspondence with validated psychometric scores. These findings suggest that personality inference from naturalistic dialogue is feasible but depends strongly on modeling strategy, trait domain, and the availability of supervised fine-tuning with validated personality measures.

Subjects

MeSH terms (MEDLINE)

Large Language ModelsPersonalityInterviews as TopicHumansFemaleMaleAdult

Author keywords

Big Five personality traitsLLMsLoRA fine-tuninglarge language modelslow-rank adaptationpersonality inferencesemistructured interviews

Authors and institutions

  • Department of Computer Science, Kent State University, 241 Mathematical Sciences Building, Kent, OH, 44242, United States, 1 3306729107.
  • Department of Psychological Sciences, Kent State University, Kent, OH, United States.

Funding

  • NIMH NIH HHS

Cite

APA

Zhu, J., Maharjan, J., Li, X., Coifman, K. G., & Jin, R. (2026). Personality Inference From Semistructured Interviews Using Large Language Models: Comparative Evaluation Study. Journal of Medical Internet Research, 28, e87129-e87129. https://doi.org/10.2196/87129

Vancouver

Zhu J, Maharjan J, Li X, Coifman KG, Jin R. Personality Inference From Semistructured Interviews Using Large Language Models: Comparative Evaluation Study. Journal of Medical Internet Research. 2026;28:e87129-e87129. doi:10.2196/87129

BibTeX
@article{Zhu2026,
  author = {Zhu, Jianfeng and Maharjan, Julina and Li, Xinyu and Coifman, Karin G and Jin, Ruoming},
  title = {Personality Inference From Semistructured Interviews Using Large Language Models: Comparative Evaluation Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e87129--e87129},
  doi = {10.2196/87129}
}
Start for free →