Preempting flaky tests via non-idempotent-outcome tests

A Wei, P Yi, Z Li, T Xie, D Marinov, W Lam - Proceedings of the 44th …, 2022 - dl.acm.org
Regression testing can greatly help in software development, but it can be seriously
undermined by flaky tests, which can both pass and fail, seemingly nondeterministically, on …

Evaluating features for machine learning detection of order-and non-order-dependent flaky tests

O Parry, GM Kapfhammer, M Hilton… - 2022 IEEE Conference …, 2022 - ieeexplore.ieee.org
Flaky tests are test cases that can pass or fail without code changes. They often waste the
time of software developers and obstruct the use of continuous integration. Previous work …

Evaluating classifiers in SE research: the ECSER pipeline and two replication studies

D Dell'Anna, FB Aydemir, F Dalpiaz - Empirical Software Engineering, 2023 - Springer
Context Automated classifiers, often based on machine learning (ML), are increasingly used
in software engineering (SE) for labelling previously unseen SE data. Researchers have …

Test flakiness across programming languages

K Barbosa, R Ferreira, G Pinto… - IEEE Transactions …, 2022 - ieeexplore.ieee.org
Regression Testing (RT) is a quality-assurance practice commonly adopted in the software
industry to check if functionality remains intact after code changes. Test flakiness is a serious …

Flakime: laboratory-controlled test flakiness impact assessment

M Cordy, R Rwemalika, A Franci, M Papadakis… - Proceedings of the 44th …, 2022 - dl.acm.org
Much research on software testing makes an implicit assumption that test failures are
deterministic such that they always witness the presence of the same defects. However, this …

Peeler: Learning to effectively predict flakiness without running tests

Y Qin, S Wang, K Liu, B Lin, H Wu, L Li… - 2022 IEEE …, 2022 - ieeexplore.ieee.org
Regression testing is a widely adopted approach to expose change-induced bugs as well as
to verify the correctness/robustness of code in modern software development settings …

Static test flakiness prediction: How far can we go?

V Pontillo, F Palomba, F Ferrucci - Empirical Software Engineering, 2022 - Springer
Test flakiness is a phenomenon occurring when a test case is non-deterministic and exhibits
both a passing and failing behavior when run against the same code. Over the last years …

Predicting flaky tests categories using few-shot learning

A Akli, G Haben, S Habchi, M Papadakis… - arXiv preprint arXiv …, 2022 - arxiv.org
Flaky tests are tests that yield different outcomes when run on the same version of a
program. This non-deterministic behaviour plagues continuous integration with false signals …

Toward static test flakiness prediction: A feasibility study

V Pontillo, F Palomba, F Ferrucci - … of the 5th International Workshop on …, 2021 - dl.acm.org
Flaky tests are tests that exhibit both a passing and failing behavior when run against the
same code. While the research community has attempted to define automated approaches …

The impact of flaky tests on historical test prioritization on chrome

E Fallahzadeh, PC Rigby - … of the 44th International Conference on …, 2022 - dl.acm.org
Test prioritization algorithms prioritize probable failing tests to give faster feedback to
developers in case a failure occurs. Test prioritization approaches that use historical failures …