Evolution-aware detection of order-dependent flaky tests

C Li, A Shi - Proceedings of the 31st ACM SIGSOFT International …, 2022 - dl.acm.org
Regression testing is an important part of the software development process but suffers from
the presence of flaky tests. Flaky tests are tests that can nondeterministically pass or fail …

iDFlakies: A framework for detecting and partially classifying flaky tests

W Lam, R Oei, A Shi, D Marinov… - 2019 12th ieee …, 2019 - ieeexplore.ieee.org
Regression testing is increasingly important with the wide use of continuous integration. A
desirable requirement for regression testing is that a test failure reliably indicates a problem …

iFixFlakies: A framework for automatically fixing order-dependent flaky tests

A Shi, W Lam, R Oei, T Xie, D Marinov - … of the 2019 27th ACM Joint …, 2019 - dl.acm.org
Regression testing provides important pass or fail signals that developers use to make
decisions after code changes. However, flaky tests, which pass or fail even when the code …

Systematically producing test orders to detect order-dependent flaky tests

C Li, MM Khosravi, W Lam, A Shi - Proceedings of the 32nd ACM …, 2023 - dl.acm.org
Software testing suffers from the presence of flaky tests, which can pass or fail when run on
the same version of code. Order-dependent tests (OD tests) are flaky tests whose outcome …

Probabilistic and systematic coverage of consecutive test-method pairs for detecting order-dependent flaky tests

A Wei, P Yi, T Xie, D Marinov, W Lam - … for the Construction and Analysis of …, 2021 - Springer
Software developers frequently check their code changes by running a set of tests against
their code. Tests that can nondeterministically pass or fail when run on the same code …

On the use of mutation in injecting test order-dependency

S Habchi, M Cordy, M Papadakis, YL Traon - arXiv preprint arXiv …, 2021 - arxiv.org
Background: Test flakiness is identified as a major issue that compromises the regression
testing process of complex software systems. Flaky tests manifest non-deterministic …

Understanding reproducibility and characteristics of flaky tests through test reruns in Java projects

W Lam, S Winter, A Astorga, V Stodden… - 2020 IEEE 31st …, 2020 - ieeexplore.ieee.org
Flaky tests are tests that can non-deterministically pass and fail. They pose a major
impediment to regression testing, because they provide an inconclusive assessment on …

iPFlakies: A framework for detecting and fixing python order-dependent flaky tests

R Wang, Y Chen, W Lam - Proceedings of the ACM/IEEE 44th …, 2022 - dl.acm.org
Developers typically run tests after code changes. Flaky tests, which are tests that can
nondeterministically pass and fail when run on the same version of code, can mislead …

A multi-factor approach for flaky test detection and automated root cause analysis

A Ahmad, FG de Oliveira Neto, Z Shi… - 2021 28th Asia …, 2021 - ieeexplore.ieee.org
Developers often spend time to determine whether test case failures are real failures or
flaky. The flaky tests, also known as non-deterministic tests, switch their outcomes without …

Evaluating features for machine learning detection of order-and non-order-dependent flaky tests

O Parry, GM Kapfhammer, M Hilton… - 2022 IEEE Conference …, 2022 - ieeexplore.ieee.org
Flaky tests are test cases that can pass or fail without code changes. They often waste the
time of software developers and obstruct the use of continuous integration. Previous work …