Are we evaluating rigorously? benchmarking recommendation for reproducible evaluation and fair comparison

Z Sun, D Yu, H Fang, J Yang, X Qu, J Zhang… - Proceedings of the 14th …, 2020 - dl.acm.org
With tremendous amount of recommendation algorithms proposed every year, one critical
issue has attracted a considerable amount of attention: there are no effective benchmarks for …

A Revisiting Study of Appropriate Offline Evaluation for Top-N Recommendation Algorithms

WX Zhao, Z Lin, Z Feng, P Wang, JR Wen - ACM Transactions on …, 2022 - dl.acm.org
In recommender systems, top-N recommendation is an important task with implicit feedback
data. Although the recent success of deep learning largely pushes forward the research on …

Elliot: A comprehensive and rigorous framework for reproducible recommender systems evaluation

VW Anelli, A Bellogín, A Ferrara, D Malitesta… - Proceedings of the 44th …, 2021 - dl.acm.org
Recommender Systems have shown to be an effective way to alleviate the over-choice
problem and provide accurate and tailored recommendations. However, the impressive …

The datasets dilemma: How much do we really know about recommendation datasets?

JY Chin, Y Chen, G Cong - … Conference on Web Search and Data …, 2022 - dl.acm.org
There has been sustained interest from both academia and industry throughout the years
due to the importance and practicability of recommendation systems. However, several …

When fairness meets bias: a debiased framework for fairness aware top-n recommendation

J Tang, S Shen, Z Wang, Z Gong, J Zhang… - Proceedings of the 17th …, 2023 - dl.acm.org
Fairness in the recommendation domain has recently attracted increasing attention due to
more and more concerns about the algorithm discrimination and ethics. While recent years …

Bars: Towards open benchmarking for recommender systems

J Zhu, Q Dai, L Su, R Ma, J Liu, G Cai, X Xiao… - Proceedings of the 45th …, 2022 - dl.acm.org
The past two decades have witnessed the rapid development of personalized
recommendation techniques. Despite the significant progress made in both research and …

Lightrec: A memory and search-efficient recommender system

D Lian, H Wang, Z Liu, J Lian, E Chen… - Proceedings of The Web …, 2020 - dl.acm.org
Deep recommender systems have achieved remarkable improvements in recent years.
Despite its superior ranking precision, the running efficiency and memory consumption turn …

KuaiRec: A fully-observed dataset and insights for evaluating recommender systems

C Gao, S Li, W Lei, J Chen, B Li, P Jiang, X He… - Proceedings of the 31st …, 2022 - dl.acm.org
The progress of recommender systems is hampered mainly by evaluation as it requires real-
time interactions between humans and systems, which is too laborious and expensive. This …

recommenderlab: an R framework for developing and testing recommendation algorithms

M Hahsler - arXiv preprint arXiv:2205.12371, 2022 - arxiv.org
Algorithms that create recommendations based on observed data have significant
commercial value for online retailers and many other industries. Recommender systems …

Rethinking the recommender research ecosystem: reproducibility, openness, and lenskit

MD Ekstrand, M Ludwig, JA Konstan… - Proceedings of the fifth …, 2011 - dl.acm.org
Recommender systems research is being slowed by the difficulty of replicating and
comparing research results. Published research uses various experimental methodologies …