Abstract Context: A Machine Learning based System (MLS) is a software system including
one or more components that learn how to perform a task from a given data set. The …

Context: Advancements in machine learning (ML) lead to a shift from the traditional view of
software development, where algorithms are hard-coded by humans, to ML systems …

This paper provides a comprehensive survey of techniques for testing machine learning
systems; Machine Learning Testing (ML testing) research. It covers 144 papers on testing …

The open-world deployment of Machine Learning (ML) algorithms in safety-critical
applications such as autonomous vehicles needs to address a variety of ML vulnerabilities …

Authors' addresses: Sina Mohseni, smohseni@ nvidia. com; Zhiding Yu, zhidingy@ nvidia.
com; Chaowei Xiao, chaoweix@ nvidia. com; Jay Yadawa, jyadawa@ nvidia. com, NVIDIA …

Automatic speech recognition (ASR) systems are ubiquitous parts of modern life. It can be
found in our smartphones, desktops, and smart home systems. To ensure its correctness in …

Multi-label Image Classification Systems (MICSs) developed based on Deep Neural
Networks (DNNs) are extensively used in people's daily life. Currently, although there are a …

Abstract Machine learning (ML) models have gained significant attention in a variety of
applications, from computer vision to natural language processing, and are almost always …

Context Assessing the accuracy in operation of a Machine Learning (ML) system for image
classification on arbitrary (unlabeled) inputs is hard. This is due to the oracle problem, which …

We aim to conduct a systematic mapping in the area of testing ML programs. We identify,
analyze and classify the existing literature to provide an overview of the area. We followed …