A survey of flaky tests

O Parry, GM Kapfhammer, M Hilton… - ACM Transactions on …, 2021 - dl.acm.org
Tests that fail inconsistently, without changes to the code under test, are described as flaky.
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …

On determinism of game engines used for simulation-based autonomous vehicle verification

G Chance, A Ghobrial, K McAreavey… - IEEE Transactions …, 2022 - ieeexplore.ieee.org
Game engines are increasingly used as simulation platforms by the autonomous vehicle
community to develop vehicle control systems and test environments. A key requirement for …

Flakeflagger: Predicting flakiness without rerunning tests

A Alshammari, C Morris, M Hilton… - 2021 IEEE/ACM 43rd …, 2021 - ieeexplore.ieee.org
When developers make changes to their code, they typically run regression tests to detect if
their recent changes (re) introduce any bugs. However, many tests are flaky, and their …

Capbug-a framework for automatic bug categorization and prioritization using nlp and machine learning algorithms

HA Ahmed, NZ Bawany, JA Shamsi - IEEE Access, 2021 - ieeexplore.ieee.org
Bug reports facilitate software development teams in improving the quality of software.
These reports include significant information related to problems encountered within a …

A survey on how test flakiness affects developers and what support they need to address it

M Gruber, G Fraser - 2022 IEEE Conference on Software …, 2022 - ieeexplore.ieee.org
Non-deterministically passing and failing test cases, so-called flaky tests, have recently
become a focus area of software engineering research. While this research focus has been …

Peeler: Learning to effectively predict flakiness without running tests

Y Qin, S Wang, K Liu, B Lin, H Wu, L Li… - 2022 IEEE …, 2022 - ieeexplore.ieee.org
Regression testing is a widely adopted approach to expose change-induced bugs as well as
to verify the correctness/robustness of code in modern software development settings …

Nature-based prediction model of bug reports based on Ensemble Machine Learning Model

SA Alsaedi, AY Noaman, AAA Gad-Elrab… - IEEE Access, 2023 - ieeexplore.ieee.org
In software development systems, the maintenance process of software systems attracted
the attention of researchers due to its importance in fixing the defects discovered in the …

[HTML][HTML] Test flakiness' causes, detection, impact and responses: A multivocal review

A Tahir, S Rasheed, J Dietrich, N Hashemi… - Journal of Systems and …, 2023 - Elsevier
Flaky tests (tests with non-deterministic outcomes) pose a major challenge for software
testing. They are known to cause significant issues, such as reducing the effectiveness and …

What we talk about when we talk about software test flakiness

M Barboni, A Bertolino, G De Angelis - … on the Quality of Information and …, 2021 - Springer
Software test flakiness is drawing increasing interest among both academic researchers and
practitioners. In this work we report our findings from a scoping review of white and grey …

Practical flaky test prediction using common code evolution and test history data

M Gruber, M Heine, N Oster… - … IEEE Conference on …, 2023 - ieeexplore.ieee.org
Non-deterministically behaving test cases cause developers to lose trust in their regression
test suites and to eventually ignore failures. Detecting flaky tests is therefore a crucial task in …