Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice
Recent years have seen growing interest among both researchers and practitioners in user-
engaged approaches to algorithm auditing, which directly engage users in detecting …
engaged approaches to algorithm auditing, which directly engage users in detecting …
Toward User-Driven Algorithm Auditing: Investigating users' strategies for uncovering harmful algorithmic behavior
A DeVos, A Dhabalia, H Shen, K Holstein… - Proceedings of the 2022 …, 2022 - dl.acm.org
Recent work in HCI suggests that users can be powerful in surfacing harmful algorithmic
behaviors that formal auditing approaches fail to detect. However, it is not well understood …
behaviors that formal auditing approaches fail to detect. However, it is not well understood …
Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work together to Surface Algorithmic Harms?
R Li, S Kingsley, C Fan, P Sinha, N Wai, J Lee… - Proceedings of the …, 2023 - dl.acm.org
Recent years have witnessed an interesting phenomenon in which users come together to
interrogate potentially harmful algorithmic behaviors they encounter in their everyday lives …
interrogate potentially harmful algorithmic behaviors they encounter in their everyday lives …
Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing
Rising concern for the societal implications of artificial intelligence systems has inspired a
wave of academic and journalistic literature in which deployed systems are audited for harm …
wave of academic and journalistic literature in which deployed systems are audited for harm …
Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
S Costanza-Chock, ID Raji, J Buolamwini - Proceedings of the 2022 …, 2022 - dl.acm.org
Algorithmic audits (or 'AI audits') are an increasingly popular mechanism for algorithmic
accountability; however, they remain poorly defined. Without a clear understanding of audit …
accountability; however, they remain poorly defined. Without a clear understanding of audit …
End-user audits: A system empowering communities to lead large-scale investigations of harmful algorithmic behavior
Because algorithm audits are conducted by technical experts, audits are necessarily limited
to the hypotheses that experts think to test. End users hold the promise to expand this …
to the hypotheses that experts think to test. End users hold the promise to expand this …
Outsider oversight: Designing a third party audit ecosystem for ai governance
Much attention has focused on algorithmic audits and impact assessments to hold
developers and users of algorithmic systems accountable. But existing algorithmic …
developers and users of algorithmic systems accountable. But existing algorithmic …
Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors
A growing body of literature has proposed formal approaches to audit algorithmic systems
for biased and harmful behaviors. While formal auditing approaches have been greatly …
for biased and harmful behaviors. While formal auditing approaches have been greatly …
A framework for assurance audits of algorithmic systems
An increasing number of regulations propose 'AI audits' as a mechanism for achieving
transparency and accountability for artificial intelligence (AI) systems. Despite some …
transparency and accountability for artificial intelligence (AI) systems. Despite some …
The algorithm audit: Scoring the algorithms that score us
In recent years, the ethical impact of AI has been increasingly scrutinized, with public
scandals emerging over biased outcomes, lack of transparency, and the misuse of data. This …
scandals emerging over biased outcomes, lack of transparency, and the misuse of data. This …