Multimodal learning is a framework for building models that make predictions based on
different types of modalities. Important challenges in multimodal learning are the inference of …

We investigate deep generative models that can exchange multiple modalities bi-
directionally, eg, generating images from corresponding texts and vice versa. Recently …

Learning generative models that span multiple data modalities, such as vision and
language, is often motivated by the desire to learn more useful, generalisable …

Multimodal variational autoencoders (VAEs) have shown promise as efficient generative
models for weakly-supervised data. Yet, despite their advantage of weak supervision, they …

Learning from different data types is a long-standing goal in machine learning research, as
multiple information sources co-occur when describing natural phenomena. However …

Multimodal representation learning, which aims to narrow the heterogeneity gap among
different modalities, plays an indispensable role in the utilization of ubiquitous multimodal …

Learning multimodal representations is a fundamentally complex research problem due to
the presence of multiple heterogeneous sources of information. Although the presence of …

Deep learning has been successfully applied to multimodal representation learning
problems, with a common strategy to learning joint representations that are shared across …

Prompt tuning has become a new paradigm for model tuning and it has demonstrated
success in natural language pretraining and even vision pretraining. In this work, we explore …

In this paper, we address the problem of conditional modality learning, whereby one is
interested in generating one modality given the other. While it is straightforward to learn a …