Black-box access is insufficient for rigorous ai audits
External audits of AI systems are increasingly recognized as a key mechanism for AI
governance. The effectiveness of an audit, however, depends on the degree of access …
governance. The effectiveness of an audit, however, depends on the degree of access …
Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Audits are critical mechanisms for identifying the risks and limitations of deployed artificial
intelligence (AI) systems. However, the effective execution of AI audits remains incredibly …
intelligence (AI) systems. However, the effective execution of AI audits remains incredibly …
AI auditing: The broken bus on the road to AI accountability
One of the most concrete measures towards meaningful AI accountability is to
consequentially assess and report the systems' performance and impact. However, the …
consequentially assess and report the systems' performance and impact. However, the …
Auditing work: Exploring the New York City algorithmic bias audit regime
L Groves, J Metcalf, A Kennedy, B Vecchione… - The 2024 ACM …, 2024 - dl.acm.org
In July 2023, New York City (NYC) implemented the first attempt to create an algorithm
auditing regime for commercial machine-learning systems. Local Law 144 (LL 144), requires …
auditing regime for commercial machine-learning systems. Local Law 144 (LL 144), requires …
Building and auditing fair algorithms: A case study in candidate screening
Academics, activists, and regulators are increasingly urging companies to develop and
deploy sociotechnical systems that are fair and unbiased. Achieving this goal, however, is …
deploy sociotechnical systems that are fair and unbiased. Achieving this goal, however, is …
Making it possible for the auditing of ai: A systematic review of ai audits and ai auditability
Artificial intelligence (AI) technologies have become the key driver of innovation in society.
However, numerous vulnerabilities of AI systems can lead to negative consequences for …
However, numerous vulnerabilities of AI systems can lead to negative consequences for …
Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing
Rising concern for the societal implications of artificial intelligence systems has inspired a
wave of academic and journalistic literature in which deployed systems are audited for harm …
wave of academic and journalistic literature in which deployed systems are audited for harm …
Outsider oversight: Designing a third party audit ecosystem for ai governance
Much attention has focused on algorithmic audits and impact assessments to hold
developers and users of algorithmic systems accountable. But existing algorithmic …
developers and users of algorithmic systems accountable. But existing algorithmic …
Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
S Costanza-Chock, ID Raji, J Buolamwini - Proceedings of the 2022 …, 2022 - dl.acm.org
Algorithmic audits (or 'AI audits') are an increasingly popular mechanism for algorithmic
accountability; however, they remain poorly defined. Without a clear understanding of audit …
accountability; however, they remain poorly defined. Without a clear understanding of audit …
Toward trustworthy AI development: mechanisms for supporting verifiable claims
With the recent wave of progress in artificial intelligence (AI) has come a growing awareness
of the large-scale impacts of AI systems, and recognition that existing regulations and norms …
of the large-scale impacts of AI systems, and recognition that existing regulations and norms …