An empirical study of flaky tests in python

M Gruber, S Lukasczyk, F Kroiß… - 2021 14th IEEE …, 2021 - ieeexplore.ieee.org
Tests that cause spurious failures without any code changes, ie, flaky tests, hamper
regression testing, increase maintenance costs, may shadow real bugs, and decrease trust …

Understanding flaky tests: The developer's perspective

M Eck, F Palomba, M Castelluccio… - Proceedings of the 2019 …, 2019 - dl.acm.org
Flaky tests are software tests that exhibit a seemingly random outcome (pass or fail) despite
exercising unchanged code. In this work, we examine the perceptions of software …

De-flake your tests: Automatically locating root causes of flaky tests in code at google

C Ziftci, D Cavalcanti - 2020 IEEE International Conference on …, 2020 - ieeexplore.ieee.org
Regression testing is a critical part of software development and maintenance. It ensures
that modifications to existing software do not break existing behavior and functionality. One …

iDFlakies: A framework for detecting and partially classifying flaky tests

W Lam, R Oei, A Shi, D Marinov… - 2019 12th ieee …, 2019 - ieeexplore.ieee.org
Regression testing is increasingly important with the wide use of continuous integration. A
desirable requirement for regression testing is that a test failure reliably indicates a problem …

A qualitative study on the sources, impacts, and mitigation strategies of flaky tests

S Habchi, G Haben, M Papadakis… - … IEEE Conference on …, 2022 - ieeexplore.ieee.org
Test flakiness forms a major testing concern. Flaky tests manifest non-deterministic
outcomes that cripple continu-ous integration and lead developers to investigate false alerts …

Understanding reproducibility and characteristics of flaky tests through test reruns in Java projects

W Lam, S Winter, A Astorga, V Stodden… - 2020 IEEE 31st …, 2020 - ieeexplore.ieee.org
Flaky tests are tests that can non-deterministically pass and fail. They pose a major
impediment to regression testing, because they provide an inconclusive assessment on …

A study on the lifecycle of flaky tests

W Lam, K Muşlu, H Sajnani… - Proceedings of the ACM …, 2020 - dl.acm.org
During regression testing, developers rely on the pass or fail outcomes of tests to check
whether changes broke existing functionality. Thus, flaky tests, which nondeterministically …

A survey on how test flakiness affects developers and what support they need to address it

M Gruber, G Fraser - 2022 IEEE Conference on Software …, 2022 - ieeexplore.ieee.org
Non-deterministically passing and failing test cases, so-called flaky tests, have recently
become a focus area of software engineering research. While this research focus has been …

Root causing flaky tests in a large-scale industrial setting

W Lam, P Godefroid, S Nath, A Santhiar… - Proceedings of the 28th …, 2019 - dl.acm.org
In today's agile world, developers often rely on continuous integration pipelines to help build
and validate their changes by executing tests in an efficient manner. One of the significant …

An empirical analysis of UI-based flaky tests

A Romano, Z Song, S Grandhi… - 2021 IEEE/ACM 43rd …, 2021 - ieeexplore.ieee.org
Flaky tests have gained attention from the research community in recent years and with
good reason. These tests lead to wasted time and resources, and they reduce the reliability …