Hierarchical semantic correspondence networks for video paragraph grounding
Abstract Video Paragraph Grounding (VPG) is an essential yet challenging task in vision-
language understanding, which aims to jointly localize multiple events from an untrimmed …
language understanding, which aims to jointly localize multiple events from an untrimmed …
Arondight: Red teaming large vision language models with auto-generated multi-modal jailbreak prompts
Large Vision Language Models (VLMs) extend and enhance the perceptual abilities of
Large Language Models (LLMs). Despite offering new possibilities for LLM applications …
Large Language Models (LLMs). Despite offering new possibilities for LLM applications …
Rethinking Multimodal Entity and Relation Extraction from a Translation Point of View
We revisit the multimodal entity and relation extraction from a translation point of view.
Special attention is paid on the misalignment issue in text-image datasets which may …
Special attention is paid on the misalignment issue in text-image datasets which may …
Vaquita: Enhancing alignment in llm-assisted video understanding
Recent advancements in language-model-based video understanding have been
progressing at a remarkable pace, spurred by the introduction of Large Language Models …
progressing at a remarkable pace, spurred by the introduction of Large Language Models …
A multi-modal context reasoning approach for conditional inference on joint textual and visual clues
Conditional inference on joint textual and visual clues is a multi-modal reasoning task that
textual clues provide prior permutation or external knowledge, which are complementary …
textual clues provide prior permutation or external knowledge, which are complementary …
Weakly-supervised learning of visual relations in multimodal pretraining
Recent work in vision-and-language pretraining has investigated supervised signals from
object detection data to learn better, fine-grained multimodal representations. In this work …
object detection data to learn better, fine-grained multimodal representations. In this work …
EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models
The recent rapid development of Large Vision-Language Models (LVLMs) has indicated
their potential for embodied tasks. However, the critical skill of spatial understanding in …
their potential for embodied tasks. However, the critical skill of spatial understanding in …
M2conceptbase: A fine-grained aligned multi-modal conceptual knowledge base
Large multi-modal models (LMMs) have demonstrated promising intelligence owing to the
rapid development of pre-training techniques. However, their fine-grained cross-modal …
rapid development of pre-training techniques. However, their fine-grained cross-modal …
DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learning
Vision-and-Language navigation (VLN) requires an agent to navigate in unseen
environment by following natural language instruction. For task completion, the agent needs …
environment by following natural language instruction. For task completion, the agent needs …
ConcEPT: Concept-Enhanced Pre-Training for Language Models
Pre-trained language models (PLMs) have been prevailing in state-of-the-art methods for
natural language processing, and knowledge-enhanced PLMs are further proposed to …
natural language processing, and knowledge-enhanced PLMs are further proposed to …