Sugarcrepe: Fixing hackable benchmarks for vision-language compositionality
In the last year alone, a surge of new benchmarks to measure $\textit {compositional} $
understanding of vision-language models have permeated the machine learning ecosystem …
understanding of vision-language models have permeated the machine learning ecosystem …
Finematch: Aspect-based fine-grained image and text mismatch detection and correction
Recent progress in large-scale pre-training has led to the development of advanced vision-
language models (VLMs) with remarkable proficiency in comprehending and generating …
language models (VLMs) with remarkable proficiency in comprehending and generating …
International Scientific Report on the Safety of Advanced AI (Interim Report)
Y Bengio, S Mindermann, D Privitera… - arXiv preprint arXiv …, 2024 - arxiv.org
This is the interim publication of the first International Scientific Report on the Safety of
Advanced AI. The report synthesises the scientific understanding of general-purpose AI--AI …
Advanced AI. The report synthesises the scientific understanding of general-purpose AI--AI …
An enhanced prompt-based LLM reasoning scheme via knowledge graph-integrated collaboration
Y Li, R Zhang, J Liu - International Conference on Artificial Neural …, 2024 - Springer
Abstract While Large Language Models (LLMs) demonstrate exceptional performance in a
multitude of Natural Language Processing (NLP) tasks, they encounter challenges in …
multitude of Natural Language Processing (NLP) tasks, they encounter challenges in …
Tailoring self-rationalizers with multi-reward distillation
Large language models (LMs) are capable of generating free-text rationales to aid question
answering. However, prior work 1) suggests that useful self-rationalization is emergent only …
answering. However, prior work 1) suggests that useful self-rationalization is emergent only …
Intentionqa: A benchmark for evaluating purchase intention comprehension abilities of language models in e-commerce
Enhancing Language Models'(LMs) ability to understand purchase intentions in E-
commerce scenarios is crucial for their effective assistance in various downstream tasks …
commerce scenarios is crucial for their effective assistance in various downstream tasks …
Knowledge editing on black-box large language models
Knowledge editing (KE) aims to efficiently and precisely modify the behavior of large
language models (LLMs) to update specific knowledge without negatively influencing other …
language models (LLMs) to update specific knowledge without negatively influencing other …
CANDLE: iterative conceptualization and instantiation distillation from large language models for commonsense reasoning
The sequential process of conceptualization and instantiation is essential to generalizable
commonsense reasoning as it allows the application of existing knowledge to unfamiliar …
commonsense reasoning as it allows the application of existing knowledge to unfamiliar …
MARS: Benchmarking the metaphysical reasoning abilities of language models with a multi-task evaluation dataset
To enable Large Language Models (LLMs) to function as conscious agents with
generalizable reasoning capabilities, it is crucial that they possess the reasoning ability to …
generalizable reasoning capabilities, it is crucial that they possess the reasoning ability to …
Kocommongen v2: A benchmark for navigating korean commonsense reasoning challenges in large language models
The evolution of large language models (LLMs) has culminated in a multitask model
paradigm where prompts drive the generation of user-specific outputs. However, this …
paradigm where prompts drive the generation of user-specific outputs. However, this …