With the emergence of various information access systems exhibiting increasing complexity,
there is a critical need for sound and scalable means of automatic evaluation. To address …

Clarifying the underlying user information need by asking clarifying questions is an
important feature of modern conversational search system. However, evaluation of such …

This research aims to explore various methods for assessing user feedback in mixed-
initiative conversational search (CS) systems. While CS systems enjoy profuse …

Current practice for evaluating recommender systems typically focuses on point estimates of
user-oriented effectiveness metrics or business metrics, sometimes combined with …

CQA systems aim to create interactive search systems that effectively retrieve information by
interacting with users. To replicate human-to-human conversations, existing work uses …

Many current effectiveness measures incorporate simplifying assumptions about user
behavior. These assumptions prevent the measures from reflecting aspects of the search …

Session search is a complex search task that involves multiple search iterations triggered by
query reformulations. We observe a Markov chain in session search: user's judgment of …

Imagine a food recommender system--how would we check if it is\emph {causing} and
fostering unhealthy eating habits or merely reflecting users' interests? How much of a user's …

A number of key Information Access tasks--Document Retrieval, Clustering, Filtering, and
their combinations--can be seen as instances of a generic {\em document organization} …

Retrieval system effectiveness can be measured in two quite different ways: by monitoring
the behavior of users and gathering data about the ease and accuracy with which they …