Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem

S Costanza-Chock, ID Raji, J Buolamwini - Proceedings of the 2022 …, 2022 - dl.acm.org
Algorithmic audits (or 'AI audits') are an increasingly popular mechanism for algorithmic
accountability; however, they remain poorly defined. Without a clear understanding of audit …

Outsider oversight: Designing a third party audit ecosystem for ai governance

ID Raji, P Xu, C Honigsberg, D Ho - Proceedings of the 2022 AAAI/ACM …, 2022 - dl.acm.org
Much attention has focused on algorithmic audits and impact assessments to hold
developers and users of algorithmic systems accountable. But existing algorithmic …

Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing

ID Raji, A Smart, RN White, M Mitchell… - Proceedings of the …, 2020 - dl.acm.org
Rising concern for the societal implications of artificial intelligence systems has inspired a
wave of academic and journalistic literature in which deployed systems are audited for harm …

Auditing algorithms: On lessons learned and the risks of data minimization

G Galdon Clavell, M Martín Zamorano… - Proceedings of the …, 2020 - dl.acm.org
In this paper, we present the Algorithmic Audit (AA) of REM! X, a personalized well-being
recommendation app developed by Telefónica Innovación Alpha. The main goal of the AA …

Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice

WH Deng, B Guo, A Devrio, H Shen, M Eslami… - Proceedings of the …, 2023 - dl.acm.org
Recent years have seen growing interest among both researchers and practitioners in user-
engaged approaches to algorithm auditing, which directly engage users in detecting …

Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling

V Ojewale, R Steed, B Vecchione, A Birhane… - arXiv preprint arXiv …, 2024 - arxiv.org
Audits are critical mechanisms for identifying the risks and limitations of deployed artificial
intelligence (AI) systems. However, the effective execution of AI audits remains incredibly …

Towards algorithm auditing: a survey on managing legal, ethical and technological risks of AI, ML and associated algorithms

A Koshiyama, E Kazim, P Treleaven, P Rai, L Szpruch… - 2021 - papers.ssrn.com
Business reliance on algorithms are becoming ubiquitous, and companies are increasingly
concerned about their algorithms causing major financial or reputational damage. High …

Making it possible for the auditing of ai: A systematic review of ai audits and ai auditability

Y Li, S Goel - Information Systems Frontiers, 2024 - Springer
Artificial intelligence (AI) technologies have become the key driver of innovation in society.
However, numerous vulnerabilities of AI systems can lead to negative consequences for …

Towards a multi-stakeholder value-based assessment framework for algorithmic systems

M Yurrita, D Murray-Rust, A Balayn… - Proceedings of the 2022 …, 2022 - dl.acm.org
In an effort to regulate Machine Learning-driven (ML) systems, current auditing processes
mostly focus on detecting harmful algorithmic biases. While these strategies have proven to …

The algorithm audit: Scoring the algorithms that score us

S Brown, J Davidovic, A Hasan - Big Data & Society, 2021 - journals.sagepub.com
In recent years, the ethical impact of AI has been increasingly scrutinized, with public
scandals emerging over biased outcomes, lack of transparency, and the misuse of data. This …