Assessing the Efficacy of Test Selection, Prioritization, and Batching Strategies in the Presence of Flaky Tests and Parallel Execution at Scale

E Fallahzadeh - 2023 - spectrum.library.concordia.ca
Effective software testing is essential for successful software releases, and numerous test
optimization techniques have been proposed to enhance this process. However, existing …

Do Test and Environmental Complexity Increase Flakiness? An Empirical Study of SAP HANA

A Berndt, T Bach, S Baltes - arXiv preprint arXiv:2409.10062, 2024 - arxiv.org
Background: Test flakiness is a major problem in the software industry. Flaky tests fail
seemingly at random without changes to the code and thus impede continuous integration …

[PDF][PDF] Strategies for Mitigating Flaky Tests in Automated Environments

R Khankhoje - researchgate.net
Automated testing, a crucial aspect of software development, plays an essential role in
assuring the dependability and effectiveness of applications. Nevertheless, the presence of …

Flakiness Goes Live: Insights from Running Flaky Tests into the Field

M Barboni, A Bertolino, G De Angelis - Available at SSRN 4095579 - papers.ssrn.com
Test flakiness is a topmost concern in software test automation. While conducting pre-
deployment testing, those tests that are flagged as flaky are put aside for being either …

The effects of computational resources on flaky tests

D Silva, M Gruber, S Gokhale, E Arteca… - IEEE Transactions …, 2024 - ieeexplore.ieee.org
Flaky tests are tests that non-deterministically pass and fail in unchanged code. These tests
can be detrimental to developers' productivity. Particularly when tests run in continuous …

On the Impact of Hitting System Resource Limits on Test Flakiness

F Leinen, A Perathoner, A Pretschner - Proceedings of the 1st …, 2024 - dl.acm.org
Regression testing aims to determine whether a change to a system introduces new bugs or
can be merged safely. Flaky tests, which are tests that fail non-deterministically and …

Surveying the developer experience of flaky tests

O Parry, GM Kapfhammer, M Hilton… - Proceedings of the 44th …, 2022 - dl.acm.org
Test cases that pass and fail without changes to the code under test are known as flaky. The
past decade has seen increasing research interest in flaky tests, though little attention has …

Empirically evaluating flaky test detection techniques combining test case rerunning and machine learning models

O Parry, GM Kapfhammer, M Hilton… - Empirical Software …, 2023 - Springer
A flaky test is a test case whose outcome changes without modification to the code of the test
case or the program under test. These tests disrupt continuous integration, cause a loss of …

Do Automatic Test Generation Tools Generate Flaky Tests?

M Gruber, MF Roslan, O Parry, F Scharnböck… - Proceedings of the 46th …, 2024 - dl.acm.org
Non-deterministic test behavior, or flakiness, is common and dreaded among developers.
Researchers have studied the issue and proposed approaches to mitigate it. However, the …

Know you neighbor: Fast static prediction of test flakiness

R Verdecchia, E Cruciani, B Miranda… - IEEE Access, 2021 - ieeexplore.ieee.org
Context: Flaky tests plague regression testing in Continuous Integration environments by
slowing down change releases and wasting testing time and effort. Despite the growing …