Wikibench: Community-driven data curation for ai evaluation on wikipedia
AI tools are increasingly deployed in community contexts. However, datasets used to
evaluate AI are typically created by developers and annotators outside a given community …
evaluate AI are typically created by developers and annotators outside a given community …
The Explanation That Hits Home: The Characteristics of Verbal Explanations That Affect Human Perception in Subjective Decision-Making
Human-AI collaborative decision-making can achieve better outcomes than either party
individually. The success of this collaboration can depend on whether the human decision …
individually. The success of this collaboration can depend on whether the human decision …
Re-examining Sexism and Misogyny Classification with Annotator Attitudes
Gender-Based Violence (GBV) is an increasing problem online, but existing datasets fail to
capture the plurality of possible annotator perspectives or ensure the representation of …
capture the plurality of possible annotator perspectives or ensure the representation of …
Aggregation Artifacts in Subjective Tasks Collapse Large Language Models' Posteriors
In-context Learning (ICL) has become the primary method for performing natural language
tasks with Large Language Models (LLMs). The knowledge acquired during pre-training is …
tasks with Large Language Models (LLMs). The knowledge acquired during pre-training is …
Understanding Traits to Support Crowdworkers' Flexibility
S Dutta - 2024 - trace.tennessee.edu
Crowdworkers are drawn to the profession in part due to the flexibility it affords. However,
the current design of crowdsourcing platforms limits this flexibility. Therefore, it is important to …
the current design of crowdsourcing platforms limits this flexibility. Therefore, it is important to …