Metamorphic testing is an approach to both test case generation and test result verification.
A central element is a set of metamorphic relations, which are necessary properties of the …

This paper provides a comprehensive survey of techniques for testing machine learning
systems; Machine Learning Testing (ML testing) research. It covers 144 papers on testing …

A test oracle determines whether a test execution reveals a fault, often by comparing the
observed program output to the expected output. This is not always practical, for example …

Deep Learning (DL) has gained wide attention in recent years. Meanwhile, bugs in DL
systems can lead to serious consequences, and may even threaten human lives. As a result …

Mutation testing is a powerful methodology for evaluating test suite quality. In mutation
testing, a large number of mutants are generated and executed against the test suite to …

This paper presents TransRepair, a fully automatic approach for testing and repairing the
consistency of machine translation systems. TransRepair combines mutation with …

The increasingly wide uptake of Machine Learning (ML) has raised the significance of the
problem of tackling bias (ie, unfairness), making it a primary software engineering concern …

Metamorphic testing is a testing technique that can be used to verify the functional
correctness of software in the absence of an ideal oracle. This paper extends metamorphic …

Metamorphic Testing: Testing the Untestable Page 1 46 IEEE SOFTWARE | PUBLISHED BY
THE IEEE COMPUTER SOCIETY 0740-7459/20©2020IEEE FEATURE: AUTOMATED …

In recent years, machine translation software has increasingly been integrated into our daily
lives. People routinely use machine translation for various applications, such as describing …