Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives
Transformer, one of the latest technological advances of deep learning, has gained
prevalence in natural language processing or computer vision. Since medical imaging bear …
prevalence in natural language processing or computer vision. Since medical imaging bear …
A review of convolutional neural network architectures and their optimizations
The research advances concerning the typical architectures of convolutional neural
networks (CNNs) as well as their optimizations are analyzed and elaborated in detail in this …
networks (CNNs) as well as their optimizations are analyzed and elaborated in detail in this …
Vision transformer adapter for dense predictions
This work investigates a simple yet powerful adapter for Vision Transformer (ViT). Unlike
recent visual transformers that introduce vision-specific inductive biases into their …
recent visual transformers that introduce vision-specific inductive biases into their …
On the integration of self-attention and convolution
Convolution and self-attention are two powerful techniques for representation learning, and
they are usually considered as two peer approaches that are distinct from each other. In this …
they are usually considered as two peer approaches that are distinct from each other. In this …
Mobile-former: Bridging mobilenet and transformer
Abstract We present Mobile-Former, a parallel design of MobileNet and transformer with a
two-way bridge in between. This structure leverages the advantages of MobileNet at local …
two-way bridge in between. This structure leverages the advantages of MobileNet at local …
Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond
Vision transformers have shown great potential in various computer vision tasks owing to
their strong capability to model long-range dependency using the self-attention mechanism …
their strong capability to model long-range dependency using the self-attention mechanism …
Vitae: Vision transformer advanced by exploring intrinsic inductive bias
Transformers have shown great potential in various computer vision tasks owing to their
strong capability in modeling long-range dependency using the self-attention mechanism …
strong capability in modeling long-range dependency using the self-attention mechanism …
Hrformer: High-resolution vision transformer for dense predict
Abstract We present a High-Resolution Transformer (HRFormer) that learns high-resolution
representations for dense prediction tasks, in contrast to the original Vision Transformer that …
representations for dense prediction tasks, in contrast to the original Vision Transformer that …
Hrformer: High-resolution transformer for dense prediction
We present a High-Resolution Transformer (HRFormer) that learns high-resolution
representations for dense prediction tasks, in contrast to the original Vision Transformer that …
representations for dense prediction tasks, in contrast to the original Vision Transformer that …
Mixformer: Mixing features across windows and dimensions
While local-window self-attention performs notably in vision tasks, it suffers from limited
receptive field and weak modeling capability issues. This is mainly because it performs self …
receptive field and weak modeling capability issues. This is mainly because it performs self …