Recent advances in large language models have enabled the development of viable
generative retrieval systems. Instead of a traditional document ranking, generative retrieval …

We introduce a general information access evaluation framework that can potentially handle
summaries, ranked document lists and even multi query sessions seamlessly. Our …

This lecture is intended to serve as an introduction to Information Retrieval (IR) effectiveness
metrics and their usage in IR experiments using test collections. Evaluation metrics are …

Search engines are widely used in our daily life. Batch evaluation of the performance of
search systems to their users has always been an essential issue in the field of information …

This resource paper addresses the challenge of evaluating Information Retrieval (IR)
systems in the era of autoregressive Large Language Models (LLMs). Traditional methods …

We present a simple and generic framework for auditing a given textual conversational
system, given some samples of its conversation sessions as its input. The framework …

Web search queries without hyperlink clicks are often referred to as abandoned queries.
Understanding the reasons for abandonment is crucial for search engines in evaluating their …

ABSTRACT NTCIR-17 introduced the FairWeb-1 task, which evaluated web page rankings
in terms of both relevance and group fairness. The present study shows how their evaluation …

We propose a new method for query-oriented extractive multi-document summarization. To
enrich the information need representation of a given query, we build a co-occurrence graph …

This is an overview of the NTCIR-12 MobileClick-2 task (a sequel to 1CLICK in NTCIR-9 and
NTCIR-10). In the MobileClick task, systems are expected to output a concise summary of …