A/B testing, also referred to as online controlled experimentation or continuous
experimentation, is a form of hypothesis testing where two variants of a piece of software are …

This paper provides a survey of the emerging area of Large Language Models (LLMs) for
Software Engineering (SE). It also sets out open research challenges for the application of …

A code generation model generates code by taking a prompt from a code comment, existing
code, or a combination of both. Although code generation models (eg, GitHub Copilot) are …

Flaky tests are tests that yield different outcomes when run on the same version of a
program. This non-deterministic behaviour plagues continuous integration with false signals …

New approaches to software testing are required due to the rising complexity of today's
software applications and the rapid growth of software engineering practices. Among these …

Flaky tests are defined as tests that manifest non-deterministic behaviour by passing and
failing intermittently for the same version of the code. These tests cripple continuous …

Flaky tests (tests with non-deterministic outcomes) pose a major challenge for software
testing. They are known to cause significant issues, such as reducing the effectiveness and …

Non-deterministically behaving test cases cause developers to lose trust in their regression
test suites and to eventually ignore failures. Detecting flaky tests is therefore a crucial task in …

The opinions and perspectives of software developers are highly regarded in software
engineering research. The experience and knowledge of software practitioners are …

Flaky tests are tests that pass and fail on different executions of the same version of a
program under test. They waste valuable developer time by making developers investigate …