Llava-critic: Learning to evaluate multimodal models
We introduce LLaVA-Critic, the first open-source large multimodal model (LMM) designed as
a generalist evaluator to assess performance across a wide range of multimodal tasks …
a generalist evaluator to assess performance across a wide range of multimodal tasks …
Prometheus 2: An open source language model specialized in evaluating other language models
Proprietary LMs such as GPT-4 are often employed to assess the quality of responses from
various LMs. However, concerns including transparency, controllability, and affordability …
various LMs. However, concerns including transparency, controllability, and affordability …
LLaVA-o1: Let Vision Language Models Reason Step-by-Step
Large language models have demonstrated substantial advancements in reasoning
capabilities, particularly through inference-time scaling, as illustrated by models such as …
capabilities, particularly through inference-time scaling, as illustrated by models such as …
Vhelm: A holistic evaluation of vision language models
Current benchmarks for assessing vision-language models (VLMs) often focus on their
perception or problem-solving capabilities and neglect other critical aspects such as …
perception or problem-solving capabilities and neglect other critical aspects such as …
Recommendation with generative models
Generative models are a class of AI models capable of creating new instances of data by
learning and sampling from their statistical distributions. In recent years, these models have …
learning and sampling from their statistical distributions. In recent years, these models have …
Spiqa: A dataset for multimodal question answering on scientific papers
S Pramanick, R Chellappa, S Venugopalan - arXiv preprint arXiv …, 2024 - arxiv.org
Seeking answers to questions within long scientific research articles is a crucial area of
study that aids readers in quickly addressing their inquiries. However, existing question …
study that aids readers in quickly addressing their inquiries. However, existing question …
Aligning to thousands of preferences via system message generalization
Although humans inherently have diverse values, current large language model (LLM)
alignment methods often assume that aligning LLMs with the general public's preferences is …
alignment methods often assume that aligning LLMs with the general public's preferences is …
A Survey on LLM-as-a-Judge
Accurate and consistent evaluation is crucial for decision-making across numerous fields,
yet it remains a challenging task due to inherent subjectivity, variability, and scale. Large …
yet it remains a challenging task due to inherent subjectivity, variability, and scale. Large …
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
The rapid advancement of Large Language Models (LLMs) has driven their expanding
application across various fields. One of the most promising applications is their role as …
application across various fields. One of the most promising applications is their role as …
Benchmarking XAI Explanations with Human-Aligned Evaluations
In this paper, we introduce PASTA (Perceptual Assessment System for explanaTion of
Artificial intelligence), a novel framework for a human-centric evaluation of XAI techniques in …
Artificial intelligence), a novel framework for a human-centric evaluation of XAI techniques in …