… In this work, we propose a dense convolutional network (DCN) with self-attention for speech
… sequence Y , a self-attention layer can be implemented by using a linear layer to compute Q…

… We propose to use fully convolutional DenseNets [26] with a self-attention mechanism to …
the four convolutional layers is 3×3. We use an extra dense block with six layers, 96 feature …

… Unlike self-attention which obtains an attention map by computing affinities between self
queries and self keys, our external attention computes the relation between self queries and a …

… , which uses the concept of dense layers and connects each dense layer to all layers in a
feed-… also includes a Self-Attention mechanism to force the network to pay more attention to the …

… self-attention module and directly synthesizes the alignment matrix instead. For simplicity,
we describe the per head and per layer … Dense and V to denote vanilla dot product attention. …

… Multi-head self-attention is a layer that essentially applies multiple self-attention mechanisms
… We use the multi-head self-attention layer in combination with the recurrent convolutional …

… insert such self-attention subnets into different levels of Dense-UNet, … dense blocks and
before downsampling/upsampling layers. We do not add any self-attention subnets for first dense …

… It is similar to the ResNet-18 architecture with the difference being significantly densely
connected feature maps in the final layer of a dense block instead of a residual block. …

… by global average pooling of deep layers can guide shallow layers to learning person
localization details. Thus, we design Global Self-Attention Module in order to provide global …

… Third, a transformer encoder has 6 encoder layers, and each one consists of a multi-head
self-attention module and a feed forward network (FFN). The dimension of feed forward is …