The field of natural language processing has seen impressive progress in recent years, with
neural network models replacing many of the traditional systems. A plethora of new models …

Contextual word representations derived from large-scale neural language models are
successful across a diverse set of NLP tasks, suggesting that they encode useful and …

We introduce extreme summarization, a new single-document summarization task which
does not favor extractive strategies and calls for an abstractive modeling approach. The idea …

While neural networks have been successfully applied to many natural language processing
tasks, they come at the cost of interpretability. In this paper, we propose a general …

Neural machine translation (NMT) models learn representations containing substantial
linguistic information. However, it is not clear if such information is fully distributed or if some …

Despite the remarkable evolution of deep neural networks in natural language processing
(NLP), their interpretability remains a challenge. Previous work largely focused on what …

We examine the effects of particular orderings of sentence pairs on the on-line training of
neural machine translation (NMT). We focus on two types of such orderings:(1) ensuring that …

Transformer-based deep NLP models are trained using hundreds of millions of parameters,
limiting their applicability in computationally constrained environments. In this paper, we …

This paper investigates contextual word representation models from the lens of similarity
analysis. Given a collection of trained models, we measure the similarity of their internal …

Beam search is widely used in neural machine translation, and usually improves translation
quality compared to greedy search. It has been widely observed that, however, beam sizes …