Context: Advancements in machine learning (ML) lead to a shift from the traditional view of
software development, where algorithms are hard-coded by humans, to ML systems …

Tests that fail inconsistently, without changes to the code under test, are described as flaky.
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …

Flaky tests are software tests that exhibit a seemingly random outcome (pass or fail) despite
exercising unchanged code. In this work, we examine the perceptions of software …

In today's agile world, developers often rely on continuous integration pipelines to help build
and validate their changes by executing tests in an efficient manner. One of the significant …

During regression testing, developers rely on the pass or fail outcomes of tests to check
whether changes broke existing functionality. Thus, flaky tests, which nondeterministically …

Regression testing is increasingly important with the wide use of continuous integration. A
desirable requirement for regression testing is that a test failure reliably indicates a problem …

When developers make changes to their code, they typically run regression tests to detect if
their recent changes (re) introduce any bugs. However, many tests are flaky, and their …

A flaky test is a test that may lead to different results in different runs on a single code under
test without any change in the test code. Test flakiness is a noxious phenomenon that slows …

Flaky tests are tests whose outcomes are non-deterministic. Despite the recent research
activity on this topic, no effort has been made on understanding the vocabulary of flaky tests …

Tests that cause spurious failures without any code changes, ie, flaky tests, hamper
regression testing, increase maintenance costs, may shadow real bugs, and decrease trust …