We introduce MMMU: a new benchmark designed to evaluate multimodal models on
massive multi-discipline tasks demanding college-level subject knowledge and deliberate …

Large Vision-Language Models (LVLMs) show significant strides in general-purpose
multimodal applications such as visual dialogue and embodied navigation. However …

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable
capabilities across image, audio, video, and text understanding. The Gemini family consists …

We present Unified-IO 2 a multimodal and multi-skill unified model capable of following
novel instructions. Unified-IO 2 can use text images audio and/or videos as input and can …

One of the fundamental problems that arise in multimodal learning tasks is the disparity of
information levels between different modalities. To resolve this problem, we propose …

Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to
maintain a holistic worldview. Most current pretraining methods, however, are limited to one …

The combination of strong visual backbones and Large Language Model (LLM) reasoning
has led to Large Multimodal Models (LMMs) becoming the current standard for a wide range …

Existed pre-training methods either focus on single-modal tasks or multi-modal tasks, and
cannot effectively adapt to each other. They can only utilize single-modal data (ie text or …

Deep multimodal learning has achieved great progress in recent years. However, current
fusion approaches are static in nature, ie, they process and fuse multimodal inputs with …

To improve the accessibility of smart devices and to simplify their usage, building models
which understand user interfaces (UIs) and assist users to complete their tasks is critical …