We report our experience of using failure symptoms, such as error messages or stack traces,
to identify flaky test failures in a Continuous Integration (CI) pipeline for a large industrial …

Researchers and practitioners alike increasingly often perceive flaky tests as a major
challenge in software engineering. They spend a lot of effort trying to detect, repair, and …

Context: Test flakiness arises when test cases have a non-deterministic, intermittent
behavior that leads them to either pass or fail when run against the same code. While …

Flaky tests can pass or fail non-deterministically, without alterations to a software system.
Such tests are frequently encountered by developers and hinder the credibility of test suites …

Due to their random nature, flaky test failures are difficult to study. Without having observed a
test to both pass and fail under the same setup, it is unknown whether a test is flaky and …

Software testing plays a crucial role to guarantee a desired level of software quality. Its goal
is to ensure software products respect specified requirements, function as intended and are …

A flaky test is a test case that can pass or fail without changes to the test case code or the
code under test. They are a wide-spread problem with serious consequences for developers …

Understanding the behavior of automated tests is crucial for improving software
development practices. This study focuses on investigating the incidence of flaky tests in …