Vision-and-language models trained to match images with text can be combined with visual
explanation methods to point to the locations of specific objects in an image. Our work …

Visual grounding, which aims to build a correspondence between visual objects and their
language entities, plays a key role in cross-modal scene understanding. One promising and …

We propose a margin-based loss for tuning joint vision-language models so that their
gradient-based explanations are consistent with region-level annotations provided by …

Referring image segmentation (RIS) aims to localize the object in an image referred by a
natural language expression. Most previous studies learn RIS with a large-scale dataset …

We address the problem of phrase grounding by learning a multi-level common semantic
space shared by the textual and visual modalities. We exploit multiple levels of feature maps …

It has been established that training a box-based detector network can enhance the
localization performance of weakly supervised and unsupervised methods. Moreover, we …

Given an input image, and nothing else, our method returns the bounding boxes of objects
in the image and phrases that describe the objects. This is achieved within an open world …

A phrase grounding model receives an input image and a text phrase and outputs a suitable
localization map. We present an effective way to refine a phrase ground model by …

This work demonstrates the implementation and use of an encoder-decoder model to
perform a many-to-many mapping of video data to text captions. The many-to-many mapping …

Localizing moments in a video via natural language queries is a challenging task where
models are trained to identify the start and the end timestamps of the moment in a video …