Recently, the large language model (LLM) community has shown increasing interest in
enhancing LLMs' capability to handle extremely long documents. As various long-text …

Large language models (LLMs) have achieved dramatic proficiency over NLP tasks with
normal length. Recently, multiple studies have committed to extending the context length …

In long context scenarios, large language models (LLMs) face three main challenges: higher
computational/financial cost, longer latency, and inferior performance. Some studies reveal …

Developing Large Language Models (LLMs) with robust long-context capabilities has been
the recent research focus, resulting in the emergence of long-context LLMs proficient in …

Large language models (LLMs), despite their impressive performance in various language
tasks, are typically limited to processing texts within context-window size. This limitation has …

Large context window is a desirable feature in large language models (LLMs). However,
due to high fine-tuning costs, scarcity of long texts, and catastrophic values introduced by …

Research on Large Language Models (LLMs) has recently witnessed an increasing interest
in extending models' context size to better capture dependencies within long documents …

Typically, training LLMs with long context sizes is computationally expensive, requiring
extensive training hours and GPU resources. Existing long-context extension methods …

While recent research endeavors have concentrated on developing Large Language
Models (LLMs) with robust long-context capabilities, due to the lack of appropriate …

Large Language Models (LLMs) have demonstrated remarkable performance across
diverse tasks but are constrained by their small context window sizes. Various efforts have …