An empirical analysis of UI-based flaky tests

A Romano, Z Song, S Grandhi… - 2021 IEEE/ACM 43rd …, 2021 - ieeexplore.ieee.org
… reports from GitHub, we manually inspect the collected samples to identify the information
relevant to our research questions. In particular, we analyze the collected flaky tests by first …

An empirical study of flaky tests in python

M Gruber, S Lukasczyk, F Kroiß… - … on Software Testing …, 2021 - ieeexplore.ieee.org
… For each flaky test we provide a classification for the root cause of its flakiness, and we
further manually investigate 100 non-order-dependent flaky tests to additionally provide a fine-…

Empirical analysis of practitioners' perceptions of test flakiness factors

A Ahmad, O Leifler, K Sandahl - Software Testing, Verification …, 2021 - Wiley Online Library
… Since flaky tests are a serious concern of software … of test cases, then these percentages
matter. Our results can be compared with the study of Mozilla [21] where 100–150 flaky tests

An empirical study of flaky tests in javascript

N Hashemi, A Tahir, S Rasheed - 2022 IEEE International …, 2022 - ieeexplore.ieee.org
… In this paper, we conduct an empirical study of the state of flaky tests in JavaScript. We
investigate two aspects of flaky tests in JavaScript projects: the main causes of flaky tests in these …

Research progress of flaky tests

W Zheng, G Liu, M Zhang, X Chen… - … on Software Analysis …, 2021 - ieeexplore.ieee.org
research progress on the flaky test. We identified 31 relevant studies and summarized the
following aspects of the flaky … of flaky tests on mutation testing through empirical research and …

A study on the lifecycle of flaky tests

W Lam, K Muşlu, H Sajnani… - Proceedings of the ACM …, 2020 - dl.acm.org
tests, the developers thought that they “fixed” the flaky tests by increasing some time values
in the tests, but our empirical experiments … threats typically found in empirical studies. In this …

Root causing, detecting, and fixing flaky tests: State of the art and future roadmap

B Zolfaghari, RM Parizi, G Srivastava… - Software: Practice …, 2021 - Wiley Online Library
… while the statistics presented in Section 1 raise the need for independent research on flaky
tests… provided flaky-specific empirical studies. In Table 6, we have listed the empirical studies

A large-scale longitudinal study of flaky tests

W Lam, S Winter, A Wei, T Xie, D Marinov… - Proceedings of the ACM …, 2020 - dl.acm.org
… The topic of flaky test research is relatively new, and this study provides quantitative empirical
evidence for the hypotheses that underpin much work on this topic. Our foundational study …

What is the vocabulary of flaky tests?

G Pinto, B Miranda, S Dissanayake… - Proceedings of the 17th …, 2020 - dl.acm.org
… the test repository. Preceding this study, we have analyzed different datasets of flaky tests
[4… for distinguishing flaky test cases from non-flaky test cases in our data set. It appears in 522 …

[HTML][HTML] Empirically evaluating flaky test detection techniques combining test case rerunning and machine learning models

O Parry, GM Kapfhammer, M Hilton… - Empirical Software …, 2023 - Springer
flaky tests from victim flaky tests), we were able to reduce its time cost by 84% at the expense
of misclassifying just 8 flaky tests … A finding that many empirical studies have in common is …