Quantitative Evaluation of Personality-Driven Short Dialogue Generation for Game NPCs Based on the Five-Factor Model
Electronics · 10.05.2026 · 15 (10) · 2030
DOI: 10.3390/electronics15102030 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 10.05.2026. Verfasst von Kanon Sasaki und 3 weiteren Autor:innen. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Society for the Promotion of Science.
Abstract (Originalsprache)
Personality-driven dialogue generation is essential for creating believable non-player characters (NPCs) in games. This study aims to (1) generate short NPC-like dialogue conditioned on predefined personality traits and (2) quantitatively evaluate whether the generated dialogue accurately reflects those traits. To achieve this, we propose a framework based on the OCEAN personality model for both controlled dialogue generation and systematic evaluation of personality consistency. We construct 32 personality configurations and generate responses to five scenario-based prompts using three models: Zephyr-7b, OpenChat-3.5-0106, and an Ollama-based OpenLLaMA-3B model. Personality consistency is evaluated using two complementary approaches: classification-based metrics (precision, recall, and F1-score) and score-based aggregation that measures alignment with intended personality traits. In addition, stability is introduced to quantify variability across multiple generated responses. The results suggest that the proposed framework supports a more structured comparison between high- and low-trait configurations within this controlled automated evaluation setting. OpenChat showed the highest performance in the automated evaluation, with F1-scores of 0.893 (high-trait) and 0.900 (low-trait), and the highest aggregated score of 340.94. Zephyr demonstrated strong stability (8.21) and consistent controllability, while the Ollama-based model showed lower consistency (F1: 0.715/0.743, score: 286.99) but substantially faster generation (0.57 s per response). Human validation on a representative subset supported the broad model-level tendency that OpenChat and Zephyr conveyed personality cues more clearly than Ollama, while the difference between OpenChat and Zephyr was less clear in human judgments.
Förderung
- Japan Society for the Promotion of Science
Zitieren
APA
Sasaki, K., Kawaguchi, S., Miyano, S., & Nishide, S. (2026). Quantitative Evaluation of Personality-Driven Short Dialogue Generation for Game NPCs Based on the Five-Factor Model. Electronics, 15(10), 2030. https://doi.org/10.3390/electronics15102030
Vancouver
Sasaki K, Kawaguchi S, Miyano S, Nishide S. Quantitative Evaluation of Personality-Driven Short Dialogue Generation for Game NPCs Based on the Five-Factor Model. Electronics. 2026;15(10):2030. doi:10.3390/electronics15102030
BibTeX
@article{Sasaki2026,
author = {Sasaki, Kanon and Kawaguchi, Sota and Miyano, Sakura and Nishide, Shun},
title = {Quantitative Evaluation of Personality-Driven Short Dialogue Generation for Game NPCs Based on the Five-Factor Model},
journal = {Electronics},
year = {2026},
volume = {15},
number = {10},
pages = {2030},
doi = {10.3390/electronics15102030}
}
EUVIMED