Towards measuring and modeling" culture" in llms: A survey
We present a survey of more than 90 recent papers that aim to study cultural representation
and inclusion in large language models (LLMs). We observe that none of the studies …
and inclusion in large language models (LLMs). We observe that none of the studies …
Cvqa: Culturally-diverse multilingual visual question answering benchmark
Visual Question Answering (VQA) is an important task in multimodal AI, and it is often used
to test the ability of vision-language models to understand and reason on knowledge …
to test the ability of vision-language models to understand and reason on knowledge …
Building socio-culturally inclusive stereotype resources with community engagement
With rapid development and deployment of generative language models in global settings,
there is an urgent need to also scale our measurements of harm, not just in the number and …
there is an urgent need to also scale our measurements of harm, not just in the number and …
ViSAGe: A global-scale analysis of visual stereotypes in text-to-image generation
Recent studies have shown that Text-to-Image (T2I) model generations can reflect social
stereotypes present in the real world. However, existing approaches for evaluating …
stereotypes present in the real world. However, existing approaches for evaluating …
Socialstigmaqa: A benchmark to uncover stigma amplification in generative language models
Current datasets for unwanted social bias auditing are limited to studying protected
demographic features such as race and gender. In this work, we introduce a comprehensive …
demographic features such as race and gender. In this work, we introduce a comprehensive …
[HTML][HTML] Normad: A benchmark for measuring the cultural adaptability of large language models
Maarten Sap - Publications Maarten Sap Publications Contact About Me CV Notes/Blogposts
Applying to grad school Giving feedback for talks Notes from my 2020 Academic Job Search …
Applying to grad school Giving feedback for talks Notes from my 2020 Academic Job Search …
How are LLMs mitigating stereotyping harms? Learning from search engine studies
A Leidinger, R Rogers - Proceedings of the AAAI/ACM Conference on AI …, 2024 - ojs.aaai.org
With the widespread availability of LLMs since the release of ChatGPT and increased public
scrutiny, commercial model development appears to have focused their efforts …
scrutiny, commercial model development appears to have focused their efforts …
GPT, large language models (LLMs) and generative artificial intelligence (GAI) models in geospatial science: a systematic review
The launch of large language models (LLMs) like ChatGPT in late 2022 and the anticipated
arrival of future GPT-x iterations have marked the beginning of the generative artificial …
arrival of future GPT-x iterations have marked the beginning of the generative artificial …
[PDF][PDF] Metrics for what, metrics for whom: assessing actionability of bias evaluation metrics in NLP
This paper introduces the concept of actionability in the context of bias measures in natural
language processing (NLP). We define actionability as the degree to which a …
language processing (NLP). We define actionability as the degree to which a …
Survey of cultural awareness in language models: Text and beyond
Large-scale deployment of large language models (LLMs) in various applications, such as
chatbots and virtual assistants, requires LLMs to be culturally sensitive to the user to ensure …
chatbots and virtual assistants, requires LLMs to be culturally sensitive to the user to ensure …