FlakeSync: Automatically Repairing Async Flaky Tests

S Rahman, A Shi - Proceedings of the IEEE/ACM 46th International …, 2024 - dl.acm.org
Regression testing is an important part of the development process but suffers from the
presence of flaky tests. Flaky tests nondeterministically pass or fail when run on the same …

A study on the lifecycle of flaky tests

W Lam, K Muşlu, H Sajnani… - Proceedings of the ACM …, 2020 - dl.acm.org
During regression testing, developers rely on the pass or fail outcomes of tests to check
whether changes broke existing functionality. Thus, flaky tests, which nondeterministically …

FlakeRepro: Automated and efficient reproduction of concurrency-related flaky tests

T Leesatapornwongsa, X Ren, S Nath - … of the 30th ACM Joint European …, 2022 - dl.acm.org
Flaky tests, which can non-deterministically pass or fail on the same code, impose significant
burden on developers by providing misleading signals during regression testing. Microsoft …

Root causing flaky tests in a large-scale industrial setting

W Lam, P Godefroid, S Nath, A Santhiar… - Proceedings of the 28th …, 2019 - dl.acm.org
In today's agile world, developers often rely on continuous integration pipelines to help build
and validate their changes by executing tests in an efficient manner. One of the significant …

Shaker: a tool for detecting more flaky tests faster

M Cordeiro, D Silva, L Teixeira… - 2021 36th IEEE/ACM …, 2021 - ieeexplore.ieee.org
A test case that intermittently passes or fails when performed under the same version of
source code and test code is said to be flaky. The presence of flaky tests wastes testing time …

Flake it'till you make it: Using automated repair to induce and fix latent test flakiness

O Parry, GM Kapfhammer, M Hilton… - Proceedings of the IEEE …, 2020 - dl.acm.org
Since flaky tests pass or fail nondeterministically, without any code changes, they are an
unreliable indicator of program quality. Developers may quarantine or delete flaky tests …

What do developer-repaired flaky tests tell us about the effectiveness of automated flaky test detection?

O Parry, GM Kapfhammer, M Hilton… - Proceedings of the 3rd …, 2022 - dl.acm.org
Because they pass or fail without code changes, flaky tests cause serious problems such as
spuriously failing builds and the eroding of developers' trust in tests. Many previous …

De-flake your tests: Automatically locating root causes of flaky tests in code at google

C Ziftci, D Cavalcanti - 2020 IEEE International Conference on …, 2020 - ieeexplore.ieee.org
Regression testing is a critical part of software development and maintenance. It ensures
that modifications to existing software do not break existing behavior and functionality. One …

An empirical study of flaky tests in javascript

N Hashemi, A Tahir, S Rasheed - 2022 IEEE International …, 2022 - ieeexplore.ieee.org
Flaky tests (tests with non-deterministic outcomes) can be problematic for testing efficiency
and software reliability. Flaky tests in test suites can also significantly delay software …

Repairing order-dependent flaky tests via test generation

C Li, C Zhu, W Wang, A Shi - … of the 44th International Conference on …, 2022 - dl.acm.org
Flaky tests are tests that pass or fail nondeterministically on the same version of code. These
tests can mislead developers concerning the quality of their code changes during regression …