Text Generation aims to produce plausible and readable text in human language from input
data. The resurgence of deep learning has greatly advanced this field, in particular, with the …

Long documents such as academic articles and business reports have been the standard
format to detail out important issues and complicated subjects that require extra attention. An …

Since the proposal of transformers, these models have been limited to bounded input
lengths, because of their need to attend to every token in the input. In this work, we propose …

Rotary Position Embeddings (RoPE) have been shown to effectively encode positional
information in transformer-based language models. However, these models fail to …

Processing and reasoning over long contexts is crucial for many practical applications of
Large Language Models (LLMs), such as document comprehension and agent construction …

Dialogue is an essential part of human communication and cooperation. Existing research
mainly focuses on short dialogue scenarios in a one-on-one fashion. However, multi-person …

Abstract Automatic Text Summarization (ATS) is an important area in Natural Language
Processing (NLP) with the goal of shortening a long text into a more compact version by …

While human evaluation remains best practice for accurately judging the faithfulness of
automatically-generated summaries, few solutions exist to address the increased difficulty …

Modeling long texts has been an essential technique in the field of natural language
processing (NLP). With the ever-growing number of long documents, it is important to …

Text summarization helps readers capture salient information from documents, news,
interviews, and meetings. However, most state-of-the-art pretrained language models (LM) …