Data-driven metric development for online controlled experiments: Seven lessons learned
Online controlled experiments, also called A/B testing, have been established as the mantra
for data-driven decision making in many web-facing companies. In recent years, there are …
for data-driven decision making in many web-facing companies. In recent years, there are …
Practical aspects of sensitivity in online experimentation with user engagement metrics
Online controlled experiments, eg, A/B testing, is the state-of-the-art approach used by
modern Internet companies to improve their services based on data-driven decisions. The …
modern Internet companies to improve their services based on data-driven decisions. The …
A dirty dozen: twelve common metric interpretation pitfalls in online controlled experiments
P Dmitriev, S Gupta, DW Kim, G Vaz - Proceedings of the 23rd ACM …, 2017 - dl.acm.org
Online controlled experiments (eg, A/B tests) are now regularly used to guide product
development and accelerate innovation in software. Product ideas are evaluated as …
development and accelerate innovation in software. Product ideas are evaluated as …
Improving the sensitivity of online controlled experiments by utilizing pre-experiment data
Online controlled experiments are at the heart of making data-driven decisions at a diverse
set of companies, including Amazon, eBay, Facebook, Google, Microsoft, Yahoo, and Zynga …
set of companies, including Amazon, eBay, Facebook, Google, Microsoft, Yahoo, and Zynga …
An activity and metric model for online controlled experiments
Accurate prioritization of efforts in product and services development is critical to the success
of every company. Online controlled experiments, also known as A/B tests, enable software …
of every company. Online controlled experiments, also known as A/B tests, enable software …
Learning Metrics that Maximise Power for Accelerated A/B-Tests
O Jeunen, A Ustimenko - Proceedings of the 30th ACM SIGKDD …, 2024 - dl.acm.org
Online controlled experiments are a crucial tool to allow for confident decision-making in
technology companies. A North Star metric is defined (such as long-term revenue or user …
technology companies. A North Star metric is defined (such as long-term revenue or user …
SQR: Balancing speed, quality and risk in online experiments
Y Xu, W Duan, S Huang - Proceedings of the 24th ACM SIGKDD …, 2018 - dl.acm.org
Controlled experimentation, also called A/B testing, is widely adopted to accelerate product
innovations in the online world. However, how fast we innovate can be limited by how we …
innovations in the online world. However, how fast we innovate can be limited by how we …
Statistical inference in two-stage online controlled experiments with treatment selection and validation
Online controlled experiments, also called A/B testing, have been established as the mantra
for data-driven decision making in many web-facing companies. A/B Testing support …
for data-driven decision making in many web-facing companies. A/B Testing support …
A/B testing intuition busters: Common misunderstandings in online controlled experiments
A/B tests, or online controlled experiments, are heavily used in industry to evaluate
implementations of ideas. While the statistics behind controlled experiments are well …
implementations of ideas. While the statistics behind controlled experiments are well …
Diluted treatment effect estimation for trigger analysis in online controlled experiments
A Deng, V Hu - Proceedings of the Eighth ACM International …, 2015 - dl.acm.org
Online controlled experiments, also called A/B testing, is playing a central role in many data-
driven web-facing companies. It is well known and intuitively obvious to many practitioners …
driven web-facing companies. It is well known and intuitively obvious to many practitioners …