Transformers have revolutionized almost all natural language processing (NLP) tasks but
suffer from memory and computational complexity that scales quadratically with sequence …

Model-based reinforcement learning methods often use learning only for the purpose of
estimating an approximate dynamics model, offloading the rest of the decision-making work …

Generative pretrained models have achieved remarkable success in various domains such
as language and computer vision. Specifically, the combination of large-scale diverse …

We apply preference modeling and reinforcement learning from human feedback (RLHF) to
finetune language models to act as helpful and harmless assistants. We find this alignment …

Network architecture plays a key role in the deep learning-based computer vision system.
The widely-used convolutional neural network and transformer treat the image as a grid or …

Transformers have shown great power in time series forecasting due to their global-range
modeling ability. However, their performance can degenerate terribly on non-stationary real …

Large Language Models (LLMs) have demonstrated remarkable performance across
diverse domains, thereby prompting researchers to explore their potential for use in …

Pre-training video transformers on extra large-scale datasets is generally required to
achieve premier performance on relatively small datasets. In this paper, we show that video …

Vision transformers have shown great success due to their high model capabilities.
However, their remarkable performance is accompanied by heavy computation costs, which …

We present TensoRF, a novel approach to model and reconstruct radiance fields. Unlike
NeRF that purely uses MLPs, we model the radiance field of a scene as a 4D tensor, which …