Empirical analysis of factors and their effect on test flakiness-practitioners' perceptions

A Ahmad, O Leifler, K Sandahl - arXiv preprint arXiv:1906.00673, 2019 - arxiv.org
Developers always wish to ensure that their latest changes to the code base do not break
existing functionality. If test cases fail, they expect these failures to be connected to the …

Empirical analysis of practitioners' perceptions of test flakiness factors

A Ahmad, O Leifler, K Sandahl - Software Testing, Verification …, 2021 - Wiley Online Library
Identifying the root causes of test flakiness is one of the challenges faced by practitioners
during software testing. In other words, the testing of the software is hampered by test …

A survey on how test flakiness affects developers and what support they need to address it

M Gruber, G Fraser - 2022 IEEE Conference on Software …, 2022 - ieeexplore.ieee.org
Non-deterministically passing and failing test cases, so-called flaky tests, have recently
become a focus area of software engineering research. While this research focus has been …

What we talk about when we talk about software test flakiness

M Barboni, A Bertolino, G De Angelis - … on the Quality of Information and …, 2021 - Springer
Software test flakiness is drawing increasing interest among both academic researchers and
practitioners. In this work we report our findings from a scoping review of white and grey …

Surveying the developer experience of flaky tests

O Parry, GM Kapfhammer, M Hilton… - Proceedings of the 44th …, 2022 - dl.acm.org
Test cases that pass and fail without changes to the code under test are known as flaky. The
past decade has seen increasing research interest in flaky tests, though little attention has …

[HTML][HTML] Test flakiness' causes, detection, impact and responses: A multivocal review

A Tahir, S Rasheed, J Dietrich, N Hashemi… - Journal of Systems and …, 2023 - Elsevier
Flaky tests (tests with non-deterministic outcomes) pose a major challenge for software
testing. They are known to cause significant issues, such as reducing the effectiveness and …

Test Flakiness' Causes, Detection, Impact and Responses: A Multivocal Review

S Rasheed, A Tahir, J Dietrich, N Hashemi… - arXiv preprint arXiv …, 2022 - arxiv.org
Flaky tests (tests with non-deterministic outcomes) pose a major challenge for software
testing. They are known to cause significant issues such as reducing the effectiveness and …

An evaluation of machine learning methods for predicting flaky tests

A Ahmad - 27th Asia-Pacific Software Engineering Conference …, 2020 - diva-portal.org
The quality of the product is uncertain if the test cases change their outcome (ie, from pass to
fail or vice versa) without modifications in the codebase. Tests that change their outcome …

Static test flakiness prediction: How far can we go?

V Pontillo, F Palomba, F Ferrucci - Empirical Software Engineering, 2022 - Springer
Test flakiness is a phenomenon occurring when a test case is non-deterministic and exhibits
both a passing and failing behavior when run against the same code. Over the last years …

Understanding flaky tests: The developer's perspective

M Eck, F Palomba, M Castelluccio… - Proceedings of the 2019 …, 2019 - dl.acm.org
Flaky tests are software tests that exhibit a seemingly random outcome (pass or fail) despite
exercising unchanged code. In this work, we examine the perceptions of software …