A meta-summary of challenges in building products with ml components–collecting experiences from 4758+ practitioners

N Nahar, H Zhang, G Lewis, S Zhou… - 2023 IEEE/ACM 2nd …, 2023 - ieeexplore.ieee.org
Incorporating machine learning (ML) components into software products raises new
software-engineering challenges and exacerbates existing ones. Many researchers have …

The participatory turn in ai design: Theoretical foundations and the current state of practice

F Delgado, S Yang, M Madaio, Q Yang - … of the 3rd ACM Conference on …, 2023 - dl.acm.org
Despite the growing consensus that stakeholders affected by AI systems should participate
in their design, enormous variation and implicit disagreements exist among current …

Zeno: An interactive framework for behavioral evaluation of machine learning

ÁA Cabrera, E Fu, D Bertucci, K Holstein… - Proceedings of the …, 2023 - dl.acm.org
Machine learning models with high accuracy on test data can still produce systematic
failures, such as harmful biases and safety issues, when deployed in the real world. To …

Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice

WH Deng, N Yildirim, M Chang, M Eslami… - Proceedings of the …, 2023 - dl.acm.org
An emerging body of research indicates that ineffective cross-functional collaboration–the
interdisciplinary work done by industry practitioners across roles–represents a major barrier …

What did my AI learn? How data scientists make sense of model behavior

ÁA Cabrera, M Tulio Ribeiro, B Lee, R Deline… - ACM Transactions on …, 2023 - dl.acm.org
Data scientists require rich mental models of how AI systems behave to effectively train,
debug, and work with them. Despite the prevalence of AI analysis tools, there is no general …

fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks

S Moore, QV Liao, H Subramonyam - … of the 2023 CHI Conference on …, 2023 - dl.acm.org
To design with AI models, user experience (UX) designers must assess the fit between the
model and user needs. Based on user research, they need to contextualize the model's …

Causalvis: Visualizations for causal inference

G Guo, E Karavani, A Endert, BC Kwon - … of the 2023 CHI conference on …, 2023 - dl.acm.org
Causal inference is a statistical paradigm for quantifying causal effects using observational
data. It is a complex process, requiring multiple steps, iterations, and collaborations with …

SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks

ZJ Wang, D Munechika, S Lee, DH Chau - Extended Abstracts of the CHI …, 2024 - dl.acm.org
Computational notebooks, such as Jupyter Notebook, have become data scientists' de facto
programming environments. Many visualization researchers and practitioners have …

Model compression in practice: Lessons learned from practitioners creating on-device machine learning experiences

F Hohman, MB Kery, D Ren, D Moritz - … of the CHI Conference on Human …, 2024 - dl.acm.org
On-device machine learning (ML) promises to improve the privacy, responsiveness, and
proliferation of new, intelligent user experiences by moving ML computation onto everyday …

How Do Analysts Understand and Verify AI-Assisted Data Analyses?

K Gu, R Shang, T Althoff, C Wang… - Proceedings of the CHI …, 2024 - dl.acm.org
Data analysis is challenging as it requires synthesizing domain knowledge, statistical
expertise, and programming skills. Assistants powered by large language models (LLMs) …