Algorithm audits are powerful tools for studying black-box systems without direct knowledge
of their inner workings. While very effective in examining technical components, the method …

Because algorithm audits are conducted by technical experts, audits are necessarily limited
to the hypotheses that experts think to test. End users hold the promise to expand this …

Recent work in HCI suggests that users can be powerful in surfacing harmful algorithmic
behaviors that formal auditing approaches fail to detect. However, it is not well understood …

Algorithms are ubiquitous and critical sources of information online, increasingly acting as
gatekeepers for users accessing or sharing information about virtually any topic, including …

While algorithm audits are growing rapidly in commonality and public importance, relatively
little scholarly work has gone toward synthesizing prior work and strategizing future research …

Recent years have witnessed an interesting phenomenon in which users come together to
interrogate potentially harmful algorithmic behaviors they encounter in their everyday lives …

In this paper, we distinguish between different sorts of assessments of algorithmic systems,
describe our process of assessing such systems for ethical risk, and share some key …

Algorithm audits have increased in recent years due to a growing need to independently
assess the performance of automatically curated services that process, filter and rank the …

A growing body of literature has proposed formal approaches to audit algorithmic systems
for biased and harmful behaviors. While formal auditing approaches have been greatly …

Algorithms influence most of our daily activities, decisions, and they guide our behaviors. It
has been argued that algorithms even have a direct impact on democratic societies. Human …