Validating synthetic usage data in living lab environments
Evaluating retrieval performance without editorial relevance judgments is challenging, but
instead, user interactions can be used as relevance signals. Living labs offer a way for small …
instead, user interactions can be used as relevance signals. Living labs offer a way for small …
A living lab architecture for reproducible shared task experimentation
No existing evaluation infrastructure for shared tasks currently supports both reproducible on-
and offline experiments. In this work, we present an architecture that ties together both types …
and offline experiments. In this work, we present an architecture that ties together both types …
[PDF][PDF] Reproducible Information Retrieval Research: From Principled System-Oriented Evaluations Towards User-Oriented Experimentation
T Breuer - 2023 - duepublico2.uni-due.de
The reproducibility of earlier findings is fundamental to the empirical sciences. Even though
this circumstance is widely acknowledged, several systematic large-scale reproducibility …
this circumstance is widely acknowledged, several systematic large-scale reproducibility …
Characteristics of an online controlled experiment: preliminary results of a literature review
F Auer, M Felderer - arXiv preprint arXiv:1912.01383, 2019 - arxiv.org
arXiv:1912.01383v2 [cs.SE] 10 Dec 2019 Page 1 Characteristics of an Online Controlled
Experiment: Preliminary Results of a Literature Review Florian Auer1 and Michael Felderer1 …
Experiment: Preliminary Results of a Literature Review Florian Auer1 and Michael Felderer1 …
Evaluating the use of Brush and Tooltip for Time Series visualizations: A comparative study
S Helin, A Eklund - 2023 - diva-portal.org
This study uses a combination of user testing and analysis to evaluate the impact of brush
and tooltip on the comprehension of time series visualizations. Employing a sequential …
and tooltip on the comprehension of time series visualizations. Employing a sequential …
Experimenting with sequential allocation procedures
JMA Kruijswijk - 2021 - research.tilburguniversity.edu
In experiments that consider the use of subjects, a crucial part is deciding which treatment to
allocate to which subject–in other words, constructing the treatment allocation procedure. In …
allocate to which subject–in other words, constructing the treatment allocation procedure. In …