Multimodal foundation models: From specialists to general-purpose assistants
Neural compression is the application of neural networks and other machine learning
methods to data compression. Recent advances in statistical machine learning have opened …
methods to data compression. Recent advances in statistical machine learning have opened …
Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts
Although Large Language Models (LLMs) and Large Multimodal Models (LMMs) exhibit
impressive skills in various domains, their ability for mathematical reasoning within visual …
impressive skills in various domains, their ability for mathematical reasoning within visual …
SEED-Bench: Benchmarking Multimodal Large Language Models
Multimodal large language models (MLLMs) building upon the foundation of powerful large
language models (LLMs) have recently demonstrated exceptional capabilities in generating …
language models (LLMs) have recently demonstrated exceptional capabilities in generating …
Mm-safetybench: A benchmark for safety evaluation of multimodal large language models
The security concerns surrounding Large Language Models (LLMs) have been extensively
explored, yet the safety of Multimodal Large Language Models (MLLMs) remains …
explored, yet the safety of Multimodal Large Language Models (MLLMs) remains …
Capsfusion: Rethinking image-text data at scale
Large multimodal models demonstrate remarkable generalist ability to perform diverse
multimodal tasks in a zero-shot manner. Large-scale web-based image-text pairs contribute …
multimodal tasks in a zero-shot manner. Large-scale web-based image-text pairs contribute …
What Matters in Training a GPT4-Style Language Model with Multimodal Inputs?
Recent advancements in GPT-4V have displayed remarkable multi-modal capabilities in
processing image inputs and following open-ended instructions. Despite these …
processing image inputs and following open-ended instructions. Despite these …
Binding touch to everything: Learning unified multimodal tactile representations
The ability to associate touch with other modalities has huge implications for humans and
computational systems. However multimodal learning with touch remains challenging due to …
computational systems. However multimodal learning with touch remains challenging due to …
Mme-survey: A comprehensive survey on evaluation of multimodal llms
As a prominent direction of Artificial General Intelligence (AGI), Multimodal Large Language
Models (MLLMs) have garnered increased attention from both industry and academia …
Models (MLLMs) have garnered increased attention from both industry and academia …
Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark
Multimodal Large Language Models (MLLMs) have gained significant attention recently,
showing remarkable potential in artificial general intelligence. However, assessing the utility …
showing remarkable potential in artificial general intelligence. However, assessing the utility …
Instruct-Imagen: Image generation with multi-modal instruction
Abstract This paper presents Instruct-Imagen a model that tackles heterogeneous image
generation tasks and generalizes across unseen tasks. We introduce multi-modal instruction …
generation tasks and generalizes across unseen tasks. We introduce multi-modal instruction …