Outsider oversight: Designing a third party audit ecosystem for ai governance

ID Raji, P Xu, C Honigsberg, D Ho - Proceedings of the 2022 AAAI/ACM …, 2022 - dl.acm.org
Much attention has focused on algorithmic audits and impact assessments to hold
developers and users of algorithmic systems accountable. But existing algorithmic …

Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem

S Costanza-Chock, ID Raji, J Buolamwini - Proceedings of the 2022 …, 2022 - dl.acm.org
Algorithmic audits (or 'AI audits') are an increasingly popular mechanism for algorithmic
accountability; however, they remain poorly defined. Without a clear understanding of audit …

Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing

ID Raji, A Smart, RN White, M Mitchell… - Proceedings of the …, 2020 - dl.acm.org
Rising concern for the societal implications of artificial intelligence systems has inspired a
wave of academic and journalistic literature in which deployed systems are audited for harm …

Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice

WH Deng, B Guo, A Devrio, H Shen, M Eslami… - Proceedings of the …, 2023 - dl.acm.org
Recent years have seen growing interest among both researchers and practitioners in user-
engaged approaches to algorithm auditing, which directly engage users in detecting …

Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling

V Ojewale, R Steed, B Vecchione, A Birhane… - arXiv preprint arXiv …, 2024 - arxiv.org
Audits are critical mechanisms for identifying the risks and limitations of deployed artificial
intelligence (AI) systems. However, the effective execution of AI audits remains incredibly …

Towards a multi-stakeholder value-based assessment framework for algorithmic systems

M Yurrita, D Murray-Rust, A Balayn… - Proceedings of the 2022 …, 2022 - dl.acm.org
In an effort to regulate Machine Learning-driven (ML) systems, current auditing processes
mostly focus on detecting harmful algorithmic biases. While these strategies have proven to …

Auditing algorithms: On lessons learned and the risks of data minimization

G Galdon Clavell, M Martín Zamorano… - Proceedings of the …, 2020 - dl.acm.org
In this paper, we present the Algorithmic Audit (AA) of REM! X, a personalized well-being
recommendation app developed by Telefónica Innovación Alpha. The main goal of the AA …

The algorithm audit: Scoring the algorithms that score us

S Brown, J Davidovic, A Hasan - Big Data & Society, 2021 - journals.sagepub.com
In recent years, the ethical impact of AI has been increasingly scrutinized, with public
scandals emerging over biased outcomes, lack of transparency, and the misuse of data. This …

Black-box access is insufficient for rigorous ai audits

S Casper, C Ezell, C Siegmann, N Kolt… - The 2024 ACM …, 2024 - dl.acm.org
External audits of AI systems are increasingly recognized as a key mechanism for AI
governance. The effectiveness of an audit, however, depends on the degree of access …

Reviewable automated decision-making: A framework for accountable algorithmic systems

J Cobbe, MSA Lee, J Singh - Proceedings of the 2021 ACM conference …, 2021 - dl.acm.org
This paper introduces reviewability as a framework for improving the accountability of
automated and algorithmic decisionmaking (ADM) involving machine learning. We draw on …