Given a language model (LM), maximum probability is a poor decoding objective for open-
ended generation, because it produces short and repetitive text. On the other hand …

We demonstrate that Contrastive Decoding--a simple, computationally light, and training-
free text generation method proposed by Li et al 2022--achieves large out-of-the-box …

The dominant paradigm for neural text generation is left-to-right decoding from
autoregressive language models. Constrained or controllable generation under complex …

Given an input sequence (or prefix), modern language models often assign high
probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix; as …

Current approaches to text generation largely rely on autoregressive models and maximum
likelihood estimation. This paradigm leads to (i) diverse but low-quality samples due to …

Large pre-trained language models have repeatedly shown their ability to produce fluent
text. Yet even when starting from a prompt, generation can continue in many plausible …

Generating text with autoregressive language models (LMs) is of great importance to many
natural language processing (NLP) applications. Previous solutions for this task often …

In open-ended natural-language generation, existing text decoding methods typically
struggle to produce text which is both diverse and high-quality. Greedy and beam search are …

Pretrained language models (LMs) are susceptible to generate text with nonfactual
information. In this work, we measure and improve the factual accuracy of large-scale LMs …

Despite recent advances in natural language generation, it remains challenging to control
attributes of generated text. We propose DExperts: Decoding-time Experts, a decoding-time …