As the core building block of vision transformers, attention is a powerful tool to capture long-
range dependency. However, such power comes at a cost: it incurs a huge computation …

Dense computer vision tasks such as object detection and segmentation require effective
multi-scale feature representation for detecting or classifying objects or regions with varying …

Vision transformer has achieved impressive performance for many vision tasks. However, it
may suffer from high redundancy in capturing local features for shallow layers. Local self …

Mixup-based augmentation has been found to be effective for generalizing models during
training, especially for Vision Transformers (ViTs) since they can easily overfit. However …

We design a new family of hybrid CNN-ViT neural networks, named FasterViT, with a focus
on high image throughput for computer vision (CV) applications. FasterViT combines the …

Due to the depth degradation effect in residual connections many efficient Vision
Transformers models that rely on stacking layers for information exchange often fail to form …

Transformers have demonstrated great potential in computer vision tasks. To avoid dense
computations of self-attentions in high-resolution visual data, some recent Transformer …

Abstract Vision Transformers (ViT) serve as powerful vision models. Unlike convolutional
neural networks, which dominated vision research in previous years, vision transformers …

The recent amalgamation of transformer and convolutional designs has led to steady
improvements in accuracy and efficiency of the models. In this work, we introduce FastViT, a …

Transformers have recently shown superior performances on various vision tasks. The large,
sometimes even global, receptive field endows Transformer models with higher …