Context: Advancements in machine learning (ML) lead to a shift from the traditional view of
software development, where algorithms are hard-coded by humans, to ML systems …

Software testing involves probing into the behavior of software systems to uncover faults.
Most testing activities are complex and costly, so a practical strategy that has been adopted …

This paper provides a comprehensive survey of techniques for testing machine learning
systems; Machine Learning Testing (ML testing) research. It covers 144 papers on testing …

Mutation testing realizes the idea of using artificial defects to support testing activities.
Mutation is typically used as a way to evaluate the adequacy of test suites, to guide the …

The performance of fault localization techniques is critical to their adoption in practice. This
paper reports on an empirical study of a wide range of fault localization techniques on real …

Test-based automated program repair (APR) has attracted huge attention from both industry
and academia. Despite the significant progress made in recent studies, the overfitting …

Unfair behaviors of Machine Learning (ML) software have garnered increasing attention and
concern among software engineers. To tackle this issue, extensive research has been …

This paper presents TransRepair, a fully automatic approach for testing and repairing the
consistency of machine translation systems. TransRepair combines mutation with …

Historically, researchers have proposed and applied many techniques to reduce the cost of
mutation testing. It has become difficult to find all techniques and to understand the cost …

The increasingly wide uptake of Machine Learning (ML) has raised the significance of the
problem of tackling bias (ie, unfairness), making it a primary software engineering concern …