A survey of flaky tests
O Parry, GM Kapfhammer, M Hilton… - ACM Transactions on …, 2021 - dl.acm.org
Tests that fail inconsistently, without changes to the code under test, are described as flaky.
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …
Context-aware code change embedding for better patch correctness assessment
Despite the capability in successfully fixing more and more real-world bugs, existing
Automated Program Repair (APR) techniques are still challenged by the long-standing …
Automated Program Repair (APR) techniques are still challenged by the long-standing …
Flakify: A black-box, language model-based predictor for flaky tests
Software testing assures that code changes do not adversely affect existing functionality.
However, a test case can be flaky, ie, passing and failing across executions, even for the …
However, a test case can be flaky, ie, passing and failing across executions, even for the …
A survey of challenges in spectrum-based software fault localization
QI Sarhan, Á Beszédes - IEEE Access, 2022 - ieeexplore.ieee.org
In software debugging, fault localization is the most difficult, expensive, tedious, and time-
consuming task, particularly for large-scale software systems. This is due to the fact that it …
consuming task, particularly for large-scale software systems. This is due to the fact that it …
A survey on how test flakiness affects developers and what support they need to address it
Non-deterministically passing and failing test cases, so-called flaky tests, have recently
become a focus area of software engineering research. While this research focus has been …
become a focus area of software engineering research. While this research focus has been …
A qualitative study on the sources, impacts, and mitigation strategies of flaky tests
Test flakiness forms a major testing concern. Flaky tests manifest non-deterministic
outcomes that cripple continu-ous integration and lead developers to investigate false alerts …
outcomes that cripple continu-ous integration and lead developers to investigate false alerts …
When life gives you oranges: detecting and diagnosing intermittent job failures at mozilla
Continuous delivery of cloud systems requires constant running of jobs (build processes,
tests, etc.). One issue that plagues this continuous integration (CI) process are intermittent …
tests, etc.). One issue that plagues this continuous integration (CI) process are intermittent …
[HTML][HTML] Empirically evaluating flaky test detection techniques combining test case rerunning and machine learning models
O Parry, GM Kapfhammer, M Hilton… - Empirical Software …, 2023 - Springer
A flaky test is a test case whose outcome changes without modification to the code of the test
case or the program under test. These tests disrupt continuous integration, cause a loss of …
case or the program under test. These tests disrupt continuous integration, cause a loss of …
On the use of test smells for prediction of flaky tests
Regression testing is an important phase to deliver software with quality. However, flaky
tests hamper the evaluation of test results and can increase costs. This is because a flaky …
tests hamper the evaluation of test results and can increase costs. This is because a flaky …
Predictive comment updating with heuristics and ast-path-based neural learning: A two-phase approach
Just-in-time comment update is a promising way to reduce the burden of developers during
software maintenance and evolution. Existing approaches can be divided into two …
software maintenance and evolution. Existing approaches can be divided into two …