EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Persona-centric Metamorphic Relation Guided Robustness Evaluation for Multi-turn Dialogue Modeling

Lin Li, Xiaohua Wu, Yanbing Chen, Xiaohui Tao, Haoran Xie, Yuefeng Li

Cognitive Computation · 2026-12 · 18 (1)

DOI: 10.1007/s12559-026-10641-3 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cognitive Computation, online erschienen am 10.08.2026. Verfasst von Lin Li und 5 weiteren Autor:innen. Die Arbeit nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Beijing PARATERA Tech Corp., Ltd., China Scholarship Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Retrieval-based dialogue systems aim to select a proper response according to multi-turn conversational history. Persona-based conversation utilizes prior knowledge to maintain persona consistency, enhancing retrieval accuracy. However, reference-based evaluation relies on high-quality data annotation, which is costly and time-consuming. To address this, we discover persona-centric metamorphic relations to infer test samples from annotated data, without additional annotation cost. Benefiting from this, this work efficiently evaluates the robustness of personalized dialogue models regarding persona consistency. Specifically, we discover three types of metamorphic relations from three aspects: self-persona, partner-persona, and response, to automatically derive new test samples . Then the inherent inference relations between originals and derivatives allow for robustness evaluation. Using this evaluation methodology, our work assesses three widely used training paradigms: non-pretraining, fine-tuning after pre-training, and prompt learning, in personalized dialogue retrieval to observe whether these paradigms are more robust or exhibit the same flaws as the other two paradigms. Our experimental results, based on the three discovered metamorphic relations with consistent outputs reveal that prompt learning is more robust than training from scratch and fine-tuning. While traditional reference-based validation and natural language processing methods achieve competitively high retrieval accuracy (Hits@1 up to 87.4%), the persona consistency of dialogue retrieval systems is just 20.98% when persona descriptions are perturbed using various metamorphic relation-based transformations.

Förderung

  • National Natural Science Foundation of China
  • Beijing PARATERA Tech Corp., Ltd.
  • China Scholarship Council

Zitieren

APA

Li, L., Wu, X., Chen, Y., Tao, X., Xie, H., & Li, Y. (2026). Persona-centric Metamorphic Relation Guided Robustness Evaluation for Multi-turn Dialogue Modeling. Cognitive Computation, 18(1). https://doi.org/10.1007/s12559-026-10641-3

Vancouver

Li L, Wu X, Chen Y, Tao X, Xie H, Li Y. Persona-centric Metamorphic Relation Guided Robustness Evaluation for Multi-turn Dialogue Modeling. Cognitive Computation. 2026;18(1). doi:10.1007/s12559-026-10641-3

BibTeX
@article{Li2026,
  author = {Li, Lin and Wu, Xiaohua and Chen, Yanbing and Tao, Xiaohui and Xie, Haoran and Li, Yuefeng},
  title = {Persona-centric Metamorphic Relation Guided Robustness Evaluation for Multi-turn Dialogue Modeling},
  journal = {Cognitive Computation},
  year = {2026},
  volume = {18},
  number = {1},
  doi = {10.1007/s12559-026-10641-3}
}
Kostenlos starten →