Deep semi-supervised learning is a fast-growing field with a range of practical applications.
This paper provides a comprehensive survey on both fundamentals and recent advances in …

Question generation is an important yet challenging problem in Artificial Intelligence (AI),
which aims to generate natural and relevant questions from various input formats, eg …

Pre-trained language models (eg, BERT (Devlin et al., 2018) and its variants) have achieved
remarkable success in varieties of NLP tasks. However, these models usually consist of …

This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-
tuned for both natural language understanding and generation tasks. The model is pre …

Many natural language processing (NLP) tasks appear in dual forms, which are generally
solved by dual learning technique that models the dualities between the coupled tasks. In …

This paper presents a new sequence-to-sequence pre-training model called ProphetNet,
which introduces a novel self-supervised objective named future n-gram prediction and the …

We propose to pre-train a unified language model for both autoencoding and partially
autoregressive language modeling tasks using a novel training procedure, referred to as a …

Large pre-trained language models have been shown to store factual knowledge in their
parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks …

NLP has achieved great progress in the past decade through the use of neural models and
large labeled datasets. The dependence on abundant data prevents NLP models from being …

Despite significant progress in text generation models, a serious limitation is their tendency
to produce text that is factually inconsistent with information in the input. Recent work has …