Context: Advancements in machine learning (ML) lead to a shift from the traditional view of
software development, where algorithms are hard-coded by humans, to ML systems …

Jupyter notebooks---documents that contain live code, equations, visualizations, and
narrative text---now are among the most popular means to compute, present, discuss and …

Software testing assures that code changes do not adversely affect existing functionality.
However, a test case can be flaky, ie, passing and failing across executions, even for the …

Test case prioritization (TCP) has been widely studied in regression testing, which aims to
optimize the execution order of test cases so as to detect more faults earlier. TCP has been …

Continuous integration (CI) is a widely used practice in modern software engineering.
Unfortunately, it is also an expensive practice-Google and Mozilla estimate their CI systems …

Validation of Android apps via testing is difficult owing to the presence of flaky tests. Due to
non-deterministic execution environments, a sequence of events (a test) may lead to …

Continuous integration (CI) is the process in which code changes are automatically
integrated, built, and tested in a shared repository. In CI, developers frequently merge and …

Software developers often rely on regression testing to ensure that recent changes made to
the source code do not introduce bugs. Flaky tests, which non-deterministically pass or fail …

Test prioritization algorithms prioritize probable failing tests to give faster feedback to
developers in case a failure occurs. Test prioritization approaches that use historical failures …

We report our experience of using failure symptoms, such as error messages or stack traces,
to identify flaky test failures in a Continuous Integration (CI) pipeline for a large industrial …