A survey of flaky tests

O Parry, GM Kapfhammer, M Hilton… - ACM Transactions on …, 2021 - dl.acm.org
Tests that fail inconsistently, without changes to the code under test, are described as flaky.
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …

Research progress of flaky tests

W Zheng, G Liu, M Zhang, X Chen… - 2021 IEEE International …, 2021 - ieeexplore.ieee.org
A flaky test is a test that both passes and fails periodically without any code changes, and its
uncontrolled uncertainty will destroy the value of the test suites and even cause developers …

A qualitative study on the sources, impacts, and mitigation strategies of flaky tests

S Habchi, G Haben, M Papadakis… - … IEEE Conference on …, 2022 - ieeexplore.ieee.org
Test flakiness forms a major testing concern. Flaky tests manifest non-deterministic
outcomes that cripple continu-ous integration and lead developers to investigate false alerts …

A replication study on the usability of code vocabulary in predicting flaky tests

G Haben, S Habchi, M Papadakis… - 2021 IEEE/ACM 18th …, 2021 - ieeexplore.ieee.org
Industrial reports indicate that flaky tests are one of the primary concerns of software testing
mainly due to the false signals they provide. To deal with this issue, researchers have …

Know you neighbor: Fast static prediction of test flakiness

R Verdecchia, E Cruciani, B Miranda… - IEEE Access, 2021 - ieeexplore.ieee.org
Context: Flaky tests plague regression testing in Continuous Integration environments by
slowing down change releases and wasting testing time and effort. Despite the growing …

Evaluating features for machine learning detection of order-and non-order-dependent flaky tests

O Parry, GM Kapfhammer, M Hilton… - 2022 IEEE Conference …, 2022 - ieeexplore.ieee.org
Flaky tests are test cases that can pass or fail without code changes. They often waste the
time of software developers and obstruct the use of continuous integration. Previous work …

An empirical study of flaky tests in javascript

N Hashemi, A Tahir, S Rasheed - 2022 IEEE International …, 2022 - ieeexplore.ieee.org
Flaky tests (tests with non-deterministic outcomes) can be problematic for testing efficiency
and software reliability. Flaky tests in test suites can also significantly delay software …

Flaky test detection in Android via event order exploration

Z Dong, A Tiwari, XL Yu, A Roychoudhury - Proceedings of the 29th ACM …, 2021 - dl.acm.org
Validation of Android apps via testing is difficult owing to the presence of flaky tests. Due to
non-deterministic execution environments, a sequence of events (a test) may lead to …

Predicting flaky tests categories using few-shot learning

A Akli, G Haben, S Habchi, M Papadakis… - arXiv preprint arXiv …, 2022 - arxiv.org
Flaky tests are tests that yield different outcomes when run on the same version of a
program. This non-deterministic behaviour plagues continuous integration with false signals …

Test flakiness across programming languages

K Barbosa, R Ferreira, G Pinto… - IEEE Transactions …, 2022 - ieeexplore.ieee.org
Regression Testing (RT) is a quality-assurance practice commonly adopted in the software
industry to check if functionality remains intact after code changes. Test flakiness is a serious …