Conversational information seeking (CIS) is concerned with a sequence of interactions
between one or more users and an information system. Interactions in CIS are primarily …

Thanks to recent advances in IR and NLP, the way users interact with search engines is
evolving rapidly, with multi-turn conversations replacing traditional one-shot textual queries …

Socio-conversational systems are dialogue systems, including what are sometimes referred
to as chatbots, vocal assistants, social robots, and embodied conversational agents, that are …

Turn-level user satisfaction is one of the most important performance metrics for
conversational agents. It can be used to monitor the agent's performance and provide …

Evaluating conversational information retrieval (CIR) systems is a challenging task that
requires a significant amount of human labor for annotation. It is imperative to invest …

Dialogue systems have received increasing attention while automatically evaluating their
performance remains challenging. User satisfaction estimation (USE) has been proposed as …

Query rewriting (QR) is an increasingly important technique for reducing user friction in a
conversational AI system. User friction is caused by various reasons, including errors in …

Automatically evaluating large scale dialogue systems' response quality is a challenging
task in dialogue research. Existing automated turn-level approaches train supervised …

A large-scale conversational agent can suffer from understanding user utterances with
various ambiguities such as ASR ambiguity, intent ambiguity, and hypothesis ambiguity …

We analyse the reliability of users' explicit feedback for evaluating the quality of
conversational agents. Using data from a commercial conversational system, we analyse …