LLMs with visual inputs, ie, Vision Language Models (VLMs), have the capacity to process
state information as visual-textual prompts and respond with policy decisions in text. We …

In this work, we propose a training-free method to inject visual referring into Multimodal
Large Language Models (MLLMs) through learnable visual token optimization. We observe …

In the video-language domain, recent works in leveraging zero-shot Large Language Model-
based reasoning for video understanding have become competitive challengers to previous …

Vision language models (VLMs) have demonstrated impressive performance across a wide
range of downstream tasks. However, their proficiency in spatial reasoning remains limited …

Large vision-language models (VLMs) have become state-of-the-art for many computer
vision tasks, with in-context learning (ICL) as a popular adaptation strategy for new ones. But …

Category-agnostic pose estimation (CAPE) has traditionally relied on support images with
annotated keypoints, a process that is often cumbersome and may fail to fully capture the …

Recent research suggests that the failures of Vision-Language Models (VLMs) at visual
reasoning often stem from erroneous agreements--when semantically distinct images are …

This paper examines how the sequencing of images and text within multi-modal prompts
influences the reasoning performance of large language models (LLMs). We performed …

Large language models (LLMs) have demonstrated exceptional capabilities in text
understanding, which has paved the way for their expansion into video LLMs (Vid-LLMs) to …

This study investigates the potential of Large Language Models (LLMs) for reconstructing
and constructing the physical world solely based on textual knowledge. It explores the …