EUVIMEDObjevte EUVIMED
Crossref

User Simulation for Evaluating Information Access Systems

Krisztian Balog, ChengXiang Zhai

Foundations and Trends® in Information Retrieval · 2024-06-13 · 18 (1-2) · 1-261

DOI: 10.1561/1500000098 ↗

At a glance

Journal article in Foundations and Trends® in Information Retrieval, published online on 2024-06-13. Written by Krisztian Balog and 1 co-authors. Cited 28 times so far (Crossref); 418 references.

AbstraktBibliografické údajeCiteRelated studies

Abstract (original language)

Information access systems, such as search engines, recom-mender systems, and conversational assistants, have become integral to our daily lives as they help us satisfy our in-formation needs. However, evaluating the effectiveness of these systems presents a long-standing and complex scien-tific challenge. This challenge is rooted in the difficulty of assessing a system's overall effectiveness in assisting users to complete tasks through interactive support, and further exacerbated by the substantial variation in user behaviour and preferences. To address this challenge, user simulation emerges as a promising solution. This monograph focuses on providing a thorough under-standing of user simulation techniques designed specifically for evaluation purposes. We begin with a background of in-formation access system evaluation and explore the diverse applications of user simulation. Subsequently, we systemati-cally review the major research progress in user simulation, covering both general frameworks for designing user simu-lators, utilizing user simulation for evaluation, and specific models and algorithms for simulating user interactions with search engines, recommender systems, and conversational assistants. Realizing that user simulation is an interdisciplinary research topic, whenever possible, we attempt to establish connections with related fields, including machine learning, dialogue systems, user modeling, and economics. We end the monograph with a broad discussion of important future research directions, many of which extend beyond the evaluation of information access systems and are expected to have broader impact on how to evaluate interactive intelligent systems in general.

Cite

APA

Balog, K., & Zhai, C. (2024). User Simulation for Evaluating Information Access Systems. Foundations and Trends® in Information Retrieval, 18(1-2), 1-261. https://doi.org/10.1561/1500000098

Vancouver

Balog K, Zhai C. User Simulation for Evaluating Information Access Systems. Foundations and Trends® in Information Retrieval. 2024;18(1-2):1-261. doi:10.1561/1500000098

BibTeX
@article{Balog2024,
  author = {Balog, Krisztian and Zhai, ChengXiang},
  title = {User Simulation for Evaluating Information Access Systems},
  journal = {Foundations and Trends® in Information Retrieval},
  year = {2024},
  volume = {18},
  number = {1-2},
  pages = {1--261},
  doi = {10.1561/1500000098}
}
Start for free →