Challenges in deploying machine learning: a survey of case studies

A Paleyes, RG Urma, ND Lawrence - ACM computing surveys, 2022 - dl.acm.org
In recent years, machine learning has transitioned from a field of academic research interest
to a field capable of solving real-world business problems. However, the deployment of …

Trustworthy AI: From principles to practices

B Li, P Qi, B Liu, S Di, J Liu, J Pei, J Yi… - ACM Computing Surveys, 2023 - dl.acm.org
The rapid development of Artificial Intelligence (AI) technology has enabled the deployment
of various systems based on it. However, many current AI systems are found vulnerable to …

“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI

N Sambasivan, S Kapania, H Highfill… - proceedings of the …, 2021 - dl.acm.org
AI models are increasingly applied in high-stakes domains like health and conservation.
Data quality carries an elevated significance in high-stakes AI due to its heightened …

Jury learning: Integrating dissenting voices into machine learning models

ML Gordon, MS Lam, JS Park, K Patel… - Proceedings of the …, 2022 - dl.acm.org
Whose labels should a machine learning (ML) algorithm learn to emulate? For ML tasks
ranging from online comment toxicity to misinformation detection to medical diagnosis …

Towards accountability for machine learning datasets: Practices from software engineering and infrastructure

B Hutchinson, A Smart, A Hanna, E Denton… - Proceedings of the …, 2021 - dl.acm.org
Datasets that power machine learning are often used, shared, and reused with little visibility
into the processes of deliberation that led to their creation. As artificial intelligence systems …

Software engineering for AI-based systems: a survey

S Martínez-Fernández, J Bogner, X Franch… - ACM Transactions on …, 2022 - dl.acm.org
AI-based systems are software systems with functionalities enabled by at least one AI
component (eg, for image-, speech-recognition, and autonomous driving). AI-based systems …

Responsible AI pattern catalogue: A collection of best practices for AI governance and engineering

Q Lu, L Zhu, X Xu, J Whittle, D Zowghi… - ACM Computing …, 2024 - dl.acm.org
Responsible Artificial Intelligence (RAI) is widely considered as one of the greatest scientific
challenges of our time and is key to increase the adoption of Artificial Intelligence (AI) …

The disagreement deconvolution: Bringing machine learning performance metrics in line with reality

ML Gordon, K Zhou, K Patel, T Hashimoto… - Proceedings of the …, 2021 - dl.acm.org
Machine learning classifiers for human-facing tasks such as comment toxicity and
misinformation often score highly on metrics such as ROC AUC but are received poorly in …

Are we learning yet? a meta review of evaluation failures across machine learning

T Liao, R Taori, ID Raji, L Schmidt - Thirty-fifth Conference on …, 2021 - openreview.net
Many subfields of machine learning share a common stumbling block: evaluation. Advances
in machine learning often evaporate under closer scrutiny or turn out to be less widely …

How ai developers overcome communication challenges in a multidisciplinary team: A case study

D Piorkowski, S Park, AY Wang, D Wang… - Proceedings of the …, 2021 - dl.acm.org
The development of AI applications is a multidisciplinary effort, involving multiple roles
collaborating with the AI developers, an umbrella term we use to include data scientists and …