Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large
groups of individuals toward solving problems. Common problems approached with …

Crowdsourcing data collection from research participants recruited from online labor
markets is now common in cognitive science. We review who is in the crowd and who can …

AI models are increasingly applied in high-stakes domains like health and conservation.
Data quality carries an elevated significance in high-stakes AI due to its heightened …

Many researchers motivate explainable AI with studies showing that human-AI team
performance on decision-making tasks improves when the AI explains its recommendations …

We address a relatively under-explored aspect of human-computer interaction: people's
abilities to understand the relationship between a machine learning model's stated …

Despite the seemingly low switching and search costs of on-demand labor markets like
Amazon Mechanical Turk, we find substantial monopsony power, as measured by the …

Job differentiation gives employers market power, allowing them to pay workers less than
their marginal productivity. We estimate a differentiated jobs model using application data …

Abstract Amazon's Mechanical Turk (MTurk) is an online marketplace for work, where
Requesters post Human Intelligence Tasks (HITs) for Workers to complete for varying …

Recent research has demonstrated that cognitive biases such as the confirmation bias or the
anchoring effect can negatively affect the quality of crowdsourced data. In practice, however …

This survey provides a comprehensive overview of the landscape of crowdsourcing
research, targeted at the machine learning community. We begin with an overview of the …