EUVIMEDEUVIMED entdecken
Open Access · CC BY-ND 4.0

Understanding and Predicting User Satisfaction with Conversational Recommender Systems

Clemencia Siro, Mohammad Aliannejadi, Maarten De Rijke

ACM Transactions on Information Systems · 31.03.2024 · 42 (2) · 1-37

DOI: 10.1145/3624989 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Information Systems, online erschienen am 08.11.2023. Verfasst von Clemencia Siro und 2 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

User satisfaction depicts the effectiveness of a system from the user’s perspective. Understanding and predicting user satisfaction is vital for the design of user-oriented evaluation methods for conversational recommender systems (CRSs) . Current approaches rely on turn-level satisfaction ratings to predict a user’s overall satisfaction with CRS. These methods assume that all users perceive satisfaction similarly, failing to capture the broader dialogue aspects that influence overall user satisfaction. We investigate the effect of several dialogue aspects on user satisfaction when interacting with a CRS. To this end, we annotate dialogues based on six aspects (i.e., relevance , interestingness , understanding , task-completion , interest-arousal , and efficiency ) at the turn and dialogue levels. We find that the concept of satisfaction varies per user. At the turn level, a system’s ability to make relevant recommendations is a significant factor in satisfaction. We adopt these aspects as features for predicting response quality and user satisfaction. We achieve an F1-score of 0.80 in classifying dissatisfactory dialogues, and a Pearson’s r of 0.73 for turn-level response quality estimation, demonstrating the effectiveness of the proposed dialogue aspects in predicting user satisfaction and being able to identify dialogues where the system is failing. With this article, we release our annotated data. 1

Zitieren

APA

Siro, C., Aliannejadi, M., & Rijke, M. D. (2024). Understanding and Predicting User Satisfaction with Conversational Recommender Systems. ACM Transactions on Information Systems, 42(2), 1-37. https://doi.org/10.1145/3624989

Vancouver

Siro C, Aliannejadi M, Rijke MD. Understanding and Predicting User Satisfaction with Conversational Recommender Systems. ACM Transactions on Information Systems. 2024;42(2):1-37. doi:10.1145/3624989

BibTeX
@article{Siro2024,
  author = {Siro, Clemencia and Aliannejadi, Mohammad and Rijke, Maarten De},
  title = {Understanding and Predicting User Satisfaction with Conversational Recommender Systems},
  journal = {ACM Transactions on Information Systems},
  year = {2024},
  volume = {42},
  number = {2},
  pages = {1--37},
  doi = {10.1145/3624989}
}
Kostenlos starten →