Flatten transformer: Vision transformer using focused linear attention

D Han, X Pan, Y Han, S Song… - Proceedings of the …, 2023 - openaccess.thecvf.com
The quadratic computation complexity of self-attention has been a persistent challenge
when applying Transformer models to vision tasks. Linear attention, on the other hand, offers …

Vision transformer with deformable attention

Z Xia, X Pan, S Song, LE Li… - Proceedings of the IEEE …, 2022 - openaccess.thecvf.com
Transformers have recently shown superior performances on various vision tasks. The large,
sometimes even global, receptive field endows Transformer models with higher …

Dynamic neural networks: A survey

Y Han, G Huang, S Song, L Yang… - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
Dynamic neural network is an emerging research topic in deep learning. Compared to static
models which have fixed computational graphs and parameters at the inference stage …

Adaptive rotated convolution for rotated object detection

Y Pu, Y Wang, Z Xia, Y Han, Y Wang… - Proceedings of the …, 2023 - openaccess.thecvf.com
Rotated object detection aims to identify and locate objects in images with arbitrary
orientation. In this scenario, the oriented directions of objects vary considerably across …

QueryDet: Cascaded sparse query for accelerating high-resolution small object detection

C Yang, Z Huang, N Wang - Proceedings of the IEEE/CVF …, 2022 - openaccess.thecvf.com
While general object detection with deep learning has achieved great success in the past
few years, the performance and efficiency of detecting small objects are far from satisfactory …

What is semantic communication? A view on conveying meaning in the era of machine intelligence

Q Lan, D Wen, Z Zhang, Q Zeng, X Chen… - Journal of …, 2021 - ieeexplore.ieee.org
In the 1940s, Claude Shannon developed the information theory focusing on quantifying the
maximum data rate that can be supported by a communication channel. Guided by this …

Not all images are worth 16x16 words: Dynamic transformers for efficient image recognition

Y Wang, R Huang, S Song… - Advances in neural …, 2021 - proceedings.neurips.cc
Abstract Vision Transformers (ViT) have achieved remarkable success in large-scale image
recognition. They split every 2D image into a fixed number of patches, each of which is …

Dynamic slimmable network

C Li, G Wang, B Wang, X Liang… - Proceedings of the …, 2021 - openaccess.thecvf.com
Current dynamic networks and dynamic pruning methods have shown their promising
capability in reducing theoretical computation complexity. However, dynamic sparse …

IA-RED: Interpretability-Aware Redundancy Reduction for Vision Transformers

B Pan, R Panda, Y Jiang, Z Wang… - Advances in Neural …, 2021 - proceedings.neurips.cc
The self-attention-based model, transformer, is recently becoming the leading backbone in
the field of computer vision. In spite of the impressive success made by transformers in a …

A survey on green deep learning

J Xu, W Zhou, Z Fu, H Zhou, L Li - arXiv preprint arXiv:2111.05193, 2021 - arxiv.org
In recent years, larger and deeper models are springing up and continuously pushing state-
of-the-art (SOTA) results across various fields like natural language processing (NLP) and …