Learning from spatio-temporal data has numerous applications such as human-behavior
analysis, object tracking, video compression, and physics simulation. However, existing …

Spatio-temporal representational learning has been widely adopted in various fields such as
action recognition, video object segmentation, and action anticipation. Previous spatio …

Abstract Integrating CNNs and RNNs to capture spatiotemporal dependencies is a prevalent
strategy for spatiotemporal prediction tasks. However, the property of CNNs to learn local …

Abstract Masked Autoencoders (MAEs) learn generalizable representations for image, text,
audio, video, etc., by reconstructing masked input data from tokens of the visible data …

Effectively modeling long spatiotemporal sequences is challenging due to the need to model
complex spatial correlations and long-range temporal dependencies simultaneously …

Recent years have witnessed remarkable advances in spatiotemporal predictive learning,
incorporating auxiliary inputs, elaborate neural architectures, and sophisticated training …

Transformer-based methods have recently achieved great advancement on 2D image-
based vision tasks. For 3D video-based tasks such as action recognition, however, directly …

This paper introduces a fusion convolutional architecture for efficient learning of spatio-
temporal features in video action recognition. Unlike 2D convolutional neural networks …

We present PredRNN++, a recurrent network for spatiotemporal predictive learning. In
pursuit of a great modeling capability for short-term video dynamics, we make our network …

The success of deep neural networks generally requires a vast amount of training data to be
labeled, which is expensive and unfeasible in scale, especially for video collections. To …