Rethinking the evaluation for conversational recommendation in the era of large language models
The recent success of large language models (LLMs) has shown great potential to develop
more powerful conversational recommender systems (CRSs), which rely on natural …
more powerful conversational recommender systems (CRSs), which rely on natural …
Understanding and predicting user satisfaction with conversational recommender systems
User satisfaction depicts the effectiveness of a system from the user's perspective.
Understanding and predicting user satisfaction is vital for the design of user-oriented …
Understanding and predicting user satisfaction is vital for the design of user-oriented …
Usimagent: Large language models for simulating search users
Due to the advantages in the cost-efficiency and reproducibility, user simulation has become
a promising solution to the user-centric evaluation of information retrieval systems …
a promising solution to the user-centric evaluation of information retrieval systems …
Large Language Models and Future of Information Retrieval: Opportunities and Challenges
CX Zhai - Proceedings of the 47th International ACM SIGIR …, 2024 - dl.acm.org
Recent years have seen great success of large language models (LLMs) in performing
many natural language processing tasks with impressive performance, including tasks that …
many natural language processing tasks with impressive performance, including tasks that …
Tutorial on User Simulation for Evaluating Information Access Systems
With the emergence of various information access systems exhibiting increasing complexity,
there is a critical need for sound and scalable means of automatic evaluation. To address …
there is a critical need for sound and scalable means of automatic evaluation. To address …
Generative Information Retrieval Evaluation
In this chapter, we consider generative information retrieval evaluation from two distinct but
interrelated perspectives. First, large language models (LLMs) themselves are rapidly …
interrelated perspectives. First, large language models (LLMs) themselves are rapidly …
User Modeling and User Profiling: A Comprehensive Survey
The integration of artificial intelligence (AI) into daily life, particularly through information
retrieval and recommender systems, has necessitated advanced user modeling and …
retrieval and recommender systems, has necessitated advanced user modeling and …
Context-Driven Interactive Query Simulations Based on Generative Large Language Models
Simulating user interactions enables a more user-oriented evaluation of information retrieval
(IR) systems. While user simulations are cost-efficient and reproducible, many approaches …
(IR) systems. While user simulations are cost-efficient and reproducible, many approaches …
Identifying Breakdowns in Conversational Recommender Systems using User Simulation
We present a methodology to systematically test conversational recommender systems with
regards to conversational breakdowns. It involves examining conversations generated …
regards to conversational breakdowns. It involves examining conversations generated …
Leveraging user simulation to develop and evaluate conversational information access agents
N Bernard - Proceedings of the 17th ACM International Conference …, 2024 - dl.acm.org
We observe a change in the way users access information, that is, the rise of conversational
information access (CIA) agents. However, the automatic evaluation of these agents remains …
information access (CIA) agents. However, the automatic evaluation of these agents remains …