A survey of flaky tests
O Parry, GM Kapfhammer, M Hilton… - ACM Transactions on …, 2021 - dl.acm.org
Tests that fail inconsistently, without changes to the code under test, are described as flaky.
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …
State of practical applicability of regression testing research: A live systematic literature review
Context: Software regression testing refers to rerunning test cases after the system under
test is modified, ascertaining that the changes have not (re-) introduced failures. Not all …
test is modified, ascertaining that the changes have not (re-) introduced failures. Not all …
Empirically evaluating readily available information for regression test optimization in continuous integration
Regression test selection (RTS) and prioritization (RTP) techniques aim to reduce testing
efforts and developer feedback time after a change to the code base. Using various …
efforts and developer feedback time after a change to the code base. Using various …
A qualitative study on the sources, impacts, and mitigation strategies of flaky tests
Test flakiness forms a major testing concern. Flaky tests manifest non-deterministic
outcomes that cripple continu-ous integration and lead developers to investigate false alerts …
outcomes that cripple continu-ous integration and lead developers to investigate false alerts …
A replication study on the usability of code vocabulary in predicting flaky tests
Industrial reports indicate that flaky tests are one of the primary concerns of software testing
mainly due to the false signals they provide. To deal with this issue, researchers have …
mainly due to the false signals they provide. To deal with this issue, researchers have …
Know you neighbor: Fast static prediction of test flakiness
Context: Flaky tests plague regression testing in Continuous Integration environments by
slowing down change releases and wasting testing time and effort. Despite the growing …
slowing down change releases and wasting testing time and effort. Despite the growing …
What helped, and what did not? an evaluation of the strategies to improve continuous integration
Continuous integration (CI) is a widely used practice in modern software engineering.
Unfortunately, it is also an expensive practice-Google and Mozilla estimate their CI systems …
Unfortunately, it is also an expensive practice-Google and Mozilla estimate their CI systems …
Continuous test suite failure prediction
C Pan, M Pradel - Proceedings of the 30th ACM SIGSOFT International …, 2021 - dl.acm.org
Continuous integration advocates to run the test suite of a project frequently, eg, for every
code change committed to a shared repository. This process imposes a high computational …
code change committed to a shared repository. This process imposes a high computational …
Empirically revisiting and enhancing IR-based test-case prioritization
Test-case prioritization (TCP) aims to detect regression bugs faster via reordering the tests
run. While TCP has been studied for over 20 years, it was almost always evaluated using …
run. While TCP has been studied for over 20 years, it was almost always evaluated using …
Empirical analysis of practitioners' perceptions of test flakiness factors
Identifying the root causes of test flakiness is one of the challenges faced by practitioners
during software testing. In other words, the testing of the software is hampered by test …
during software testing. In other words, the testing of the software is hampered by test …