Software testing with large language models: Survey, landscape, and vision

J Wang, Y Huang, C Chen, Z Liu… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Pre-trained large language models (LLMs) have recently emerged as a breakthrough
technology in natural language processing and artificial intelligence, with the ability to …

A software engineering perspective on engineering machine learning systems: State of the art and challenges

G Giray - Journal of Systems and Software, 2021 - Elsevier
Context: Advancements in machine learning (ML) lead to a shift from the traditional view of
software development, where algorithms are hard-coded by humans, to ML systems …

Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models

Y Deng, CS Xia, H Peng, C Yang, L Zhang - Proceedings of the 32nd …, 2023 - dl.acm.org
Deep Learning (DL) systems have received exponential growth in popularity and have
become ubiquitous in our everyday life. Such systems are built on top of popular DL …

Large language models are edge-case fuzzers: Testing deep learning libraries via fuzzgpt

Y Deng, CS Xia, C Yang, SD Zhang, S Yang… - arXiv preprint arXiv …, 2023 - arxiv.org
Deep Learning (DL) library bugs affect downstream DL applications, emphasizing the need
for reliable systems. Generating valid input programs for fuzzing DL libraries is challenging …

A comprehensive study of deep learning compiler bugs

Q Shen, H Ma, J Chen, Y Tian, SC Cheung… - Proceedings of the 29th …, 2021 - dl.acm.org
There are increasing uses of deep learning (DL) compilers to generate optimized code,
boosting the runtime performance of DL models on specific hardware. Like their traditional …

Free lunch for testing: Fuzzing deep-learning libraries from open source

A Wei, Y Deng, C Yang, L Zhang - Proceedings of the 44th International …, 2022 - dl.acm.org
Deep learning (DL) systems can make our life much easier, and thus are gaining more and
more attention from both academia and industry. Meanwhile, bugs in DL systems can be …

Fuzzing deep-learning libraries via automated relational api inference

Y Deng, C Yang, A Wei, L Zhang - Proceedings of the 30th ACM Joint …, 2022 - dl.acm.org
Deep Learning (DL) has gained wide attention in recent years. Meanwhile, bugs in DL
systems can lead to serious consequences, and may even threaten human lives. As a result …

Fairness testing: A comprehensive survey and analysis of trends

Z Chen, JM Zhang, M Hort, F Sarro… - arXiv preprint arXiv …, 2022 - arxiv.org
Unfair behaviors of Machine Learning (ML) software have garnered increasing attention and
concern among software engineers. To tackle this issue, extensive research has been …

Nnsmith: Generating diverse and valid test cases for deep learning compilers

J Liu, J Lin, F Ruffy, C Tan, J Li, A Panda… - Proceedings of the 28th …, 2023 - dl.acm.org
Deep-learning (DL) compilers such as TVM and TensorRT are increasingly being used to
optimize deep neural network (DNN) models to meet performance, resource utilization and …

Prioritizing test inputs for deep neural networks via mutation analysis

Z Wang, H You, J Chen, Y Zhang… - 2021 IEEE/ACM 43rd …, 2021 - ieeexplore.ieee.org
Deep Neural Network (DNN) testing is one of the most widely-used ways to guarantee the
quality of DNNs. However, labeling test inputs to check the correctness of DNN prediction is …