iFixFlakies: A framework for automatically fixing order-dependent flaky tests

A Shi, W Lam, R Oei, T Xie, D Marinov - … of the 2019 27th ACM Joint …, 2019 - dl.acm.org
Regression testing provides important pass or fail signals that developers use to make
decisions after code changes. However, flaky tests, which pass or fail even when the code …

iDFlakies: A framework for detecting and partially classifying flaky tests

W Lam, R Oei, A Shi, D Marinov… - 2019 12th ieee …, 2019 - ieeexplore.ieee.org
Regression testing is increasingly important with the wide use of continuous integration. A
desirable requirement for regression testing is that a test failure reliably indicates a problem …

De-flake your tests: Automatically locating root causes of flaky tests in code at google

C Ziftci, D Cavalcanti - 2020 IEEE International Conference on …, 2020 - ieeexplore.ieee.org
Regression testing is a critical part of software development and maintenance. It ensures
that modifications to existing software do not break existing behavior and functionality. One …

Understanding reproducibility and characteristics of flaky tests through test reruns in Java projects

W Lam, S Winter, A Astorga, V Stodden… - 2020 IEEE 31st …, 2020 - ieeexplore.ieee.org
Flaky tests are tests that can non-deterministically pass and fail. They pose a major
impediment to regression testing, because they provide an inconclusive assessment on …

A study on the lifecycle of flaky tests

W Lam, K Muşlu, H Sajnani… - Proceedings of the ACM …, 2020 - dl.acm.org
During regression testing, developers rely on the pass or fail outcomes of tests to check
whether changes broke existing functionality. Thus, flaky tests, which nondeterministically …

An empirical analysis of flaky tests

Q Luo, F Hariri, L Eloussi, D Marinov - Proceedings of the 22nd ACM …, 2014 - dl.acm.org
Regression testing is a crucial part of software development. It checks that software changes
do not break existing functionality. An important assumption of regression testing is that test …

A large-scale longitudinal study of flaky tests

W Lam, S Winter, A Wei, T Xie, D Marinov… - Proceedings of the ACM …, 2020 - dl.acm.org
Flaky tests are tests that can non-deterministically pass or fail for the same code version.
These tests undermine regression testing efficiency, because developers cannot easily …

DeFlaker automatically detecting flaky tests

J Bell, O Legunsen, M Hilton, L Eloussi… - Proceedings of the 40th …, 2018 - dl.acm.org
Developers often run tests to check that their latest changes to a code repository did not
break any previously working functionality. Ideally, any new test failures would indicate …

Dependent-test-aware regression testing techniques

W Lam, A Shi, R Oei, S Zhang, MD Ernst… - Proceedings of the 29th …, 2020 - dl.acm.org
Developers typically rely on regression testing techniques to ensure that their changes do
not break existing functionality. Unfortunately, these techniques suffer from flaky tests, which …

Understanding flaky tests: The developer's perspective

M Eck, F Palomba, M Castelluccio… - Proceedings of the 2019 …, 2019 - dl.acm.org
Flaky tests are software tests that exhibit a seemingly random outcome (pass or fail) despite
exercising unchanged code. In this work, we examine the perceptions of software …