Understanding long real-world videos requires modeling of long-range visual
dependencies. To this end we explore video-first architectures building on the common …

One of the main challenges of multimodal learning is the need to combine heterogeneous
modalities (eg video audio text). For example video and audio are obtained at much higher …

An ideal model for dense video captioning--predicting captions localized temporally in a
video--should be able to handle long input videos predict rich detailed textual descriptions …

Most transformer-based video encoders are limited to short temporal contexts due to their
quadratic complexity. While various attempts have been made to extend this context, this …

Training autonomous agents that can learn new tasks from only a handful of demonstrations
is a long-standing problem in machine learning. Recently, transformers have been shown to …

Abstract Vision-Language models (VLMs) have excelled in the image-domain---especially in
zero-shot settings---thanks to the availability of vast pretraining data (ie paired image-text …

" This book provides a framework for thinking about foundational philosophical questions
surrounding machine learning as an approach to artificial intelligence. Specifically, it links …

In this work, we investigate Active Vision Reinforcement Learning (ActiveVision-RL), where
an embodied agent simultaneously learns action policy for the task while also controlling its …

Temporal action detection is a very important and challenging task in the field of video
understanding, especially for datasets with significant differences in action duration. The …

Document-level relation extraction aims to categorize the association between any two
entities within a document. We find that previous methods for document-level relation …