do not break existing functionality. An important assumption of regression testing is that test
outcomes are deterministic: an unmodified test is expected to either always pass or always
fail for the same code under test. Unfortunately, in practice, some tests often called flaky tests—
have non-deterministic outcomes. Such tests undermine the regression testing as they make
it difficult to rely on test results. We present the first extensive study of flaky tests. We study in …