The availability of large-scale image captioning and visual question answering datasets has
contributed significantly to recent successes in vision-and-language pre-training. However …

Tremendous progress has been made in recent years in developing better image captioning
models, yet most of them rely on a separate object detector to extract regional features …

Abstract We propose Localized Narratives, a new form of multimodal image annotations
connecting vision and language. We ask annotators to describe an image with their voice …

While most conversational AI systems focus on textual dialogue only, conditioning
utterances on visual context (when it's available) can lead to more realistic conversations …

A simultaneous understanding of questions and images is crucial in Visual Question
Answering (VQA). While the existing models have achieved satisfactory performance by …

In this paper, we extend scene understanding to include that of human sketch. The result is a
complete trilogy of scene representation from three diverse and complementary modalities …

We present Answer-Me, a task-aware multi-task framework which unifies a variety of
question answering tasks, such as, visual question answering, visual entailment, visual …

Image captioning is a longstanding problem in the field of computer vision and natural
language processing. To date, researchers have produced impressive state-of-the-art …

Human ratings are currently the most accurate way to assess the quality of an image
captioning model, yet most often the only used outcome of an expensive human rating …

Automatic image captioning has improved significantly over the last few years, but the
problem is far from being solved, with state of the art models still often producing low quality …