Attention mechanism is the key to large language models, and attention matrix serves as an
algorithmic and computational bottleneck for such a scheme. In this paper, we define two …

In modern machine learning, inner product attention computation is a fundamental task for
training large language models such as Transformer, GPT-1, BERT, GPT-2, GPT-3 and …

We consider supervised learning problems within the positive-definite kernel framework,
such as kernel ridge regression, kernel logistic regression or the support vector machine …

In modern machine learning, attention computation is a fundamental task for training large
language models such as Transformer, GPT-4 and ChatGPT. In this work, we study …

Square matrices appear in many machine learning problems and models. Optimization over
a large square matrix is expensive in memory and in time. Therefore an economic …

We introduce a “learning-based” algorithm for the low-rank decomposition problem: given
an $ n\times d $ matrix $ A $, and a parameter $ k $, compute a rank-$ k $ matrix $ A'$ that …

Large language models (LLMs) have brought significant and transformative changes in
human society. These models have demonstrated remarkable capabilities in natural …

We study the common problem of approximating a target matrix with a matrix of lower rank.
We provide a simple and efficient (EM) algorithm for solving weighted low-rank …

Sparse Greedy Matrix Approximation for Machine Learning | Proceedings of the Seventeenth
International Conference on Machine Learning skip to main content ACM Digital Library home …

The sparsity constrained rank-one matrix approximation problem is a difficult mathematical
optimization problem which arises in a wide array of useful applications in engineering …