Abstract Pretraining Vision Transformers (ViTs) has achieved great success in visual
recognition. A following scenario is to adapt a ViT to various image and video recognition …

Modern hierarchical vision transformers have added several vision-specific components in
the pursuit of supervised classification performance. While these components lead to …

This paper presents an efficient multi-scale vision Transformer, called ResT, that capably
served as a general-purpose backbone for image recognition. Unlike existing Transformer …

Built on top of self-attention mechanisms, vision transformers have demonstrated
remarkable performance on a variety of vision tasks recently. While achieving excellent …

There has been a debate on the choice of plain vs. hierarchical vision transformers, where
researchers often believe that the former (eg, ViT) has a simpler design but the latter (eg …

Recently, Vision Transformers (ViTs) have been broadly explored in visual recognition. With
low efficiency in encoding fine-level features, the performance of ViTs is still inferior to the …

The recently proposed Visual image Transformers (ViT) with pure attention have achieved
promising performance on image recognition tasks, such as image classification. However …

Vision Transformers (ViTs) take all the image patches as tokens and construct multi-head
self-attention (MHSA) among them. Complete leverage of these image tokens brings …

Vision Transformer (ViT) is emerging as the state-of-the-art architecture for image
recognition. While recent studies suggest that ViTs are more robust than their convolutional …

The past year has witnessed the rapid development of applying the Transformer module to
vision problems. While some researchers have demonstrated that Transformer-based …