After more than 70 years of evolution, great achievements have been made in machine
translation. Especially in recent years, translation quality has been greatly improved with the …

Training machine learning models in a meaningful order, from the easy samples to the hard
ones, using curriculum learning can provide performance improvements over the standard …

Non-autoregressive (NAR) generation, which is first proposed in neural machine translation
(NMT) to speed up inference, has attracted much attention in both machine learning and …

Abstract Knowledge distillation (KD) is the preliminary step for training non-autoregressive
translation (NAT) models, which eases the training of NAT models at the cost of losing …

Recent work on non-autoregressive neural machine translation (NAT) aims at improving the
efficiency by parallel decoding without sacrificing the quality. However, existing NAT …

Non-autoregressive neural machine translation (NAMT) has received increasing attention
recently in virtue of its promising acceleration paradigm for fast decoding. However, these …

Knowledge distillation (KD) is commonly used to construct synthetic data for training non-
autoregressive translation (NAT) models. However, there exists a discrepancy on low …

Error correction techniques have been used to refine the output sentences from automatic
speech recognition (ASR) models and achieve a lower word error rate (WER) than original …

While large scale pre-trained language models such as BERT have achieved great success
on various natural language understanding tasks, how to efficiently and effectively …

The masked language model has received remarkable attention due to its effectiveness on
various natural language processing tasks. However, few works have adopted this …