Click model-based information retrieval metrics
A Chuklin, P Serdyukov, M De Rijke - Proceedings of the 36th …, 2013 - dl.acm.org
A Chuklin, P Serdyukov, M De Rijke
Proceedings of the 36th international ACM SIGIR conference on Research and …, 2013•dl.acm.orgIn recent years many models have been proposed that are aimed at predicting clicks of web
search users. In addition, some information retrieval evaluation metrics have been built on
top of a user model. In this paper we bring these two directions together and propose a
common approach to converting any click model into an evaluation metric. We then put the
resulting model-based metrics as well as traditional metrics (like DCG or Precision) into a
common evaluation framework and compare them along a number of dimensions. One of …
search users. In addition, some information retrieval evaluation metrics have been built on
top of a user model. In this paper we bring these two directions together and propose a
common approach to converting any click model into an evaluation metric. We then put the
resulting model-based metrics as well as traditional metrics (like DCG or Precision) into a
common evaluation framework and compare them along a number of dimensions. One of …
In recent years many models have been proposed that are aimed at predicting clicks of web search users. In addition, some information retrieval evaluation metrics have been built on top of a user model. In this paper we bring these two directions together and propose a common approach to converting any click model into an evaluation metric. We then put the resulting model-based metrics as well as traditional metrics (like DCG or Precision) into a common evaluation framework and compare them along a number of dimensions.
One of the dimensions we are particularly interested in is the agreement between offline and online experimental outcomes. It is widely believed, especially in an industrial setting, that online A/B-testing and interleaving experiments are generally better at capturing system quality than offline measurements. We show that offline metrics that are based on click models are more strongly correlated with online experimental outcomes than traditional offline metrics, especially in situations when we have incomplete relevance judgements.
ACM Digital Library以上显示的是最相近的搜索结果。 查看全部搜索结果