Root causing, detecting, and fixing flaky tests: State of the art and future roadmap

B Zolfaghari, RM Parizi, G Srivastava… - Software: Practice …, 2021 - Wiley Online Library
A flaky test is a test that may lead to different results in different runs on a single code under
test without any change in the test code. Test flakiness is a noxious phenomenon that slows …

[HTML][HTML] Test flakiness' causes, detection, impact and responses: A multivocal review

A Tahir, S Rasheed, J Dietrich, N Hashemi… - Journal of Systems and …, 2023 - Elsevier
Flaky tests (tests with non-deterministic outcomes) pose a major challenge for software
testing. They are known to cause significant issues, such as reducing the effectiveness and …

Research progress of flaky tests

W Zheng, G Liu, M Zhang, X Chen… - 2021 IEEE International …, 2021 - ieeexplore.ieee.org
A flaky test is a test that both passes and fails periodically without any code changes, and its
uncontrolled uncertainty will destroy the value of the test suites and even cause developers …

Surveying the developer experience of flaky tests

O Parry, GM Kapfhammer, M Hilton… - Proceedings of the 44th …, 2022 - dl.acm.org
Test cases that pass and fail without changes to the code under test are known as flaky. The
past decade has seen increasing research interest in flaky tests, though little attention has …

A survey of flaky tests

O Parry, GM Kapfhammer, M Hilton… - ACM Transactions on …, 2021 - dl.acm.org
Tests that fail inconsistently, without changes to the code under test, are described as flaky.
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …

A survey on how test flakiness affects developers and what support they need to address it

M Gruber, G Fraser - 2022 IEEE Conference on Software …, 2022 - ieeexplore.ieee.org
Non-deterministically passing and failing test cases, so-called flaky tests, have recently
become a focus area of software engineering research. While this research focus has been …

A qualitative study on the sources, impacts, and mitigation strategies of flaky tests

S Habchi, G Haben, M Papadakis… - … IEEE Conference on …, 2022 - ieeexplore.ieee.org
Test flakiness forms a major testing concern. Flaky tests manifest non-deterministic
outcomes that cripple continu-ous integration and lead developers to investigate false alerts …

Understanding flaky tests: The developer's perspective

M Eck, F Palomba, M Castelluccio… - Proceedings of the 2019 …, 2019 - dl.acm.org
Flaky tests are software tests that exhibit a seemingly random outcome (pass or fail) despite
exercising unchanged code. In this work, we examine the perceptions of software …

What we talk about when we talk about software test flakiness

M Barboni, A Bertolino, G De Angelis - … on the Quality of Information and …, 2021 - Springer
Software test flakiness is drawing increasing interest among both academic researchers and
practitioners. In this work we report our findings from a scoping review of white and grey …

Root causing flaky tests in a large-scale industrial setting

W Lam, P Godefroid, S Nath, A Santhiar… - Proceedings of the 28th …, 2019 - dl.acm.org
In today's agile world, developers often rely on continuous integration pipelines to help build
and validate their changes by executing tests in an efficient manner. One of the significant …