Multimodal sentiment analysis aims to extract and integrate semantic information collected
from multiple modalities to recognize the expressed emotions and sentiment in multimodal …

The success of large-scale contrastive vision-language pretraining (CLIP) has benefited
both visual recognition and multimodal content understanding. The concise design brings …

Sequence-to-sequence learning with neural networks has become the de facto standard for
sequence modeling. This approach typically models the local distribution over the next …

The NLP community has seen substantial recent interest in grounding to facilitate interaction
between language technologies and the world. However, as a community, we use the term …

Learning semantically meaningful sentence embeddings is an open problem in natural
language processing. In this work, we propose a sentence embedding learning approach …

Designing better machine translation systems by considering auxiliary inputs such as
images has attracted much attention in recent years. While existing methods show promising …

Cognitive grammar suggests that the acquisition of language grammar is grounded within
visual structures. While grammar is an essential representation of natural language, it also …

We investigate video-aided grammar induction, which learns a constituency parser from
both unlabeled text and its corresponding video. Existing methods of multi-modal grammar …

We introduce a new task, unsupervised vision-language (VL) grammar induction. Given an
image-caption pair, the goal is to extract a shared hierarchical structure for both image and …

Large-scale pre-trained language models have garnered significant attention in recent years
due to their effectiveness in extracting sentence representations. However, most pre-trained …