A/B testing, also referred to as online controlled experimentation or continuous
experimentation, is a form of hypothesis testing where two variants of a piece of software are …

Context: Advancements in machine learning (ML) lead to a shift from the traditional view of
software development, where algorithms are hard-coded by humans, to ML systems …

Tests that fail inconsistently, without changes to the code under test, are described as flaky.
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …

Learning-based fault localization has been intensively studied recently. Prior studies have
shown that traditional Learning-to-Rank techniques can help precisely diagnose fault …

We present Bugs. jar, a large-scale dataset for research in automated debugging, patching,
and testing of Java programs. Bugs. jar is comprised of 1,158 bugs and patches, drawn from …

Continuous integration (CI) systems automate the compilation, building, and testing of
software. Despite CI being a widely used activity in software engineering, we do not know …

We report our experience with SapFix: the first deployment of automated end-to-end fault
fixing, from test case design through to deployed repairs in production code. We have used …

Developers often run tests to check that their latest changes to a code repository did not
break any previously working functionality. Ideally, any new test failures would indicate …

Test-based automated program repair (APR) has attracted huge attention from both industry
and academia. Despite the significant progress made in recent studies, the overfitting …

Flaky tests are software tests that exhibit a seemingly random outcome (pass or fail) despite
exercising unchanged code. In this work, we examine the perceptions of software …