This work uniquely identifies and characterizes four prevalent multimodal model
architectural patterns in the contemporary multimodal landscape. Systematically …

Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design
computer agents with intelligent capabilities such as understanding, reasoning, and learning …

Learning multimodal representations involves integrating information from multiple
heterogeneous sources of data. In order to accelerate progress towards understudied …

We present TinyLLaVA Factory, an open-source modular codebase for small-scale large
multimodal models (LMMs) with a focus on simplicity of code implementations, extensibility …

This work addresses the problem of cross-modality inference (CMI), ie, inferring missing
data of unavailable perceptual modalities (eg, sound) using data from available perceptual …

Multimodal models are expected to be a critical component to future advances in artificial
intelligence. This field is starting to grow rapidly with a surge of new design elements …

Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design
computer agents with intelligent capabilities such as understanding, reasoning, and learning …

One of the fundamental goals of machine learning is to create systems capable of
processing data from a variety of modalities such as images and text. I argue that the next …

Learning multimodal representations involves integrating information from multiple
heterogeneous sources of data. It is a challenging yet crucial area with numerous real-world …

Recent advancements in large-scale models have showcased remarkable generalization
capabilities in various tasks. However, integrating multimodal processing into these models …