We study the dynamics of gradient flow for training a multi-head softmax attention model for
in-context learning of multi-task linear regression. We establish the global convergence of …

In-context learning (ICL) is the ability of a model to learn a new task by observing a few
exemplars in its context. While prevalent in NLP, this capability has recently also been …

In-context learning (ICL) is a cornerstone of large language model (LLM) functionality, yet its
theoretical foundations remain elusive due to the complexity of transformer architectures. In …

The in-context learning (ICL) capability of pre-trained models based on the transformer
architecture has received growing interest in recent years. While theoretical understanding …

In this paper, we study multi-task structured bandit problem where the goal is to learn a near-
optimal algorithm that minimizes cumulative regret. The tasks share a common structure and …

In-context learning allows models like transformers to adapt to new tasks from a few
examples without updating their weights, a desirable trait for reinforcement learning (RL) …

Pretrained foundation models have exhibited extraordinary in-context learning performance,
allowing zero-shot generalization to new tasks not encountered during the pretraining. In the …

Building operations consume approximately 40% of global energy, with Heating, Ventilation,
and Air Conditioning (HVAC) systems responsible for up to 50% of this consumption. As …

Recent studies have demonstrated that Transformers can perform in-context reinforcement
learning (RL) by imitating a source RL algorithm. This enables them to adapt to new tasks in …

В последнее время нейросетевые модели стали одним из наиболее перспективных
направлений в области автоматического извлечения признаков из цифровых сигналов …