[HTML][HTML] A/B testing: A systematic literature review
A/B testing, also referred to as online controlled experimentation or continuous
experimentation, is a form of hypothesis testing where two variants of a piece of software are …
experimentation, is a form of hypothesis testing where two variants of a piece of software are …
Predicting user satisfaction with intelligent assistants
There is a rapid growth in the use of voice-controlled intelligent personal assistants on
mobile devices, such as Microsoft's Cortana, Google Now, and Apple's Siri. They …
mobile devices, such as Microsoft's Cortana, Google Now, and Apple's Siri. They …
[HTML][HTML] Controlled experimentation in continuous experimentation: Knowledge and challenges
Context: Continuous experimentation and A/B testing is an established industry practice that
has been researched for more than 10 years. Our aim is to synthesize the conducted …
has been researched for more than 10 years. Our aim is to synthesize the conducted …
Data-driven metric development for online controlled experiments: Seven lessons learned
Online controlled experiments, also called A/B testing, have been established as the mantra
for data-driven decision making in many web-facing companies. In recent years, there are …
for data-driven decision making in many web-facing companies. In recent years, there are …
Measuring user satisfaction on smart speaker intelligent assistants using intent sensitive query embeddings
Intelligent assistants are increasingly being used on smart speaker devices, such as
Amazon Echo, Google Home, Apple Homepod, and Harmon Kardon Invoke with Cortana …
Amazon Echo, Google Home, Apple Homepod, and Harmon Kardon Invoke with Cortana …
False discovery in A/B testing
R Berman, C Van den Bulte - Management Science, 2022 - pubsonline.informs.org
We investigate what fraction of all significant results in website A/B testing is actually null
effects (ie, the false discovery rate (FDR)). Our data consist of 4,964 effects from 2,766 …
effects (ie, the false discovery rate (FDR)). Our data consist of 4,964 effects from 2,766 …
Boosted decision tree regression adjustment for variance reduction in online controlled experiments
Nowadays, the development of most leading web services is controlled by online
experiments that qualify and quantify the steady stream of their updates achieving more than …
experiments that qualify and quantify the steady stream of their updates achieving more than …
Objective bayesian two sample hypothesis testing for online controlled experiments
A Deng - Proceedings of the 24th International Conference on …, 2015 - dl.acm.org
As A/B testing gains wider adoption in the industry, more people begin to realize the
limitations of the traditional frequentist null hypothesis statistical testing (NHST). The large …
limitations of the traditional frequentist null hypothesis statistical testing (NHST). The large …
Future user engagement prediction and its application to improve the sensitivity of online experiments
Modern Internet companies improve their services by means of data-driven decisions that
are based on online controlled experiments (also known as A/B tests). To run more online …
are based on online controlled experiments (also known as A/B tests). To run more online …
Winner's curse: Bias estimation for total effects of features in online controlled experiments
MR Lee, M Shen - Proceedings of the 24th ACM SIGKDD international …, 2018 - dl.acm.org
Online controlled experiments, or A/B testing, has been a standard framework adopted by
most online product companies to measure the effect of any new change. Companies use …
most online product companies to measure the effect of any new change. Companies use …