Evaluating features for machine learning detection of order-and non-order-dependent flaky tests

O Parry, GM Kapfhammer, M Hilton… - 2022 IEEE Conference …, 2022 - ieeexplore.ieee.org
Flaky tests are test cases that can pass or fail without code changes. They often waste the
time of software developers and obstruct the use of continuous integration. Previous work …

iDFlakies: A framework for detecting and partially classifying flaky tests

W Lam, R Oei, A Shi, D Marinov… - 2019 12th ieee …, 2019 - ieeexplore.ieee.org
Regression testing is increasingly important with the wide use of continuous integration. A
desirable requirement for regression testing is that a test failure reliably indicates a problem …

Evolution-aware detection of order-dependent flaky tests

C Li, A Shi - Proceedings of the 31st ACM SIGSOFT International …, 2022 - dl.acm.org
Regression testing is an important part of the software development process but suffers from
the presence of flaky tests. Flaky tests are tests that can nondeterministically pass or fail …

A qualitative study on the sources, impacts, and mitigation strategies of flaky tests

S Habchi, G Haben, M Papadakis… - … IEEE Conference on …, 2022 - ieeexplore.ieee.org
Test flakiness forms a major testing concern. Flaky tests manifest non-deterministic
outcomes that cripple continu-ous integration and lead developers to investigate false alerts …

Research progress of flaky tests

W Zheng, G Liu, M Zhang, X Chen… - 2021 IEEE International …, 2021 - ieeexplore.ieee.org
A flaky test is a test that both passes and fails periodically without any code changes, and its
uncontrolled uncertainty will destroy the value of the test suites and even cause developers …

iPFlakies: A framework for detecting and fixing python order-dependent flaky tests

R Wang, Y Chen, W Lam - Proceedings of the ACM/IEEE 44th …, 2022 - dl.acm.org
Developers typically run tests after code changes. Flaky tests, which are tests that can
nondeterministically pass and fail when run on the same version of code, can mislead …

Practical flaky test prediction using common code evolution and test history data

M Gruber, M Heine, N Oster… - … IEEE Conference on …, 2023 - ieeexplore.ieee.org
Non-deterministically behaving test cases cause developers to lose trust in their regression
test suites and to eventually ignore failures. Detecting flaky tests is therefore a crucial task in …

What is the vocabulary of flaky tests?

G Pinto, B Miranda, S Dissanayake… - Proceedings of the 17th …, 2020 - dl.acm.org
Flaky tests are tests whose outcomes are non-deterministic. Despite the recent research
activity on this topic, no effort has been made on understanding the vocabulary of flaky tests …

Root causing flaky tests in a large-scale industrial setting

W Lam, P Godefroid, S Nath, A Santhiar… - Proceedings of the 28th …, 2019 - dl.acm.org
In today's agile world, developers often rely on continuous integration pipelines to help build
and validate their changes by executing tests in an efficient manner. One of the significant …

Surveying the developer experience of flaky tests

O Parry, GM Kapfhammer, M Hilton… - Proceedings of the 44th …, 2022 - dl.acm.org
Test cases that pass and fail without changes to the code under test are known as flaky. The
past decade has seen increasing research interest in flaky tests, though little attention has …