Position encoding (PE), an essential part of self-attention networks (SANs), is used to
preserve the word order information for natural language processing tasks, generating fixed …

Non-autoregressive neural machine translation (NAT) generates each target word in parallel
and has achieved promising inference acceleration. However, existing NAT models still …

Pre-trained language models (PrLMs) have demonstrated superior performance due to their
strong ability to learn universal language representations from self-supervised pre-training …

Abstract In recent years, Recurrent Neural Network based Neural Machine Translation (RNN-
based NMT) equipped with an attention mechanism from the decoder to encoder, has …

Despite success in many domains, neural models struggle in settings where train and test
examples are drawn from different distributions. In particular, in contrast to humans …

Rare words are usually replaced with a single token in the current encoder–decoder style of
neural machine translation, challenging the translation modeling by an obscured context. In …

机器翻译是指通过计算机将源语言句子翻译到与之语义等价的目标语言句子的过程,
是自然语言处理领域的一个重要研究方向. 神经机器翻译仅需使用神经网络就能实现从源语言到 …

Neural machine translation (NMT) has achieved great success due to the ability to generate
high-quality sentences. Compared with human translations, one of the drawbacks of current …

In the Transformer network architecture, positional embeddings are used to encode order
dependencies into the input representation. However, this input representation only involves …

One of the reasons Transformer translation models are popular is that self-attention
networks for context modelling can be easily parallelized at sequence level. However, the …