Lora: Low-rank adaptation of large language models
An important paradigm of natural language processing consists of large-scale pre-training
on general domain data and adaptation to particular tasks or domains. As we pre-train larger …
on general domain data and adaptation to particular tasks or domains. As we pre-train larger …
Matrix completion has no spurious local minimum
Matrix completion is a basic machine learning problem that has wide applications,
especially in collaborative filtering and recommender systems. Simple non-convex …
especially in collaborative filtering and recommender systems. Simple non-convex …
No spurious local minima in nonconvex low rank problems: A unified geometric analysis
In this paper we develop a new framework that captures the common landscape underlying
the common non-convex low-rank matrix problems including matrix sensing, matrix …
the common non-convex low-rank matrix problems including matrix sensing, matrix …
Mr. biq: Post-training non-uniform quantization based on minimizing the reconstruction error
Post-training quantization compresses a neural network within few hours with only a small
unlabeled calibration set. However, so far it has been only discussed and empirically …
unlabeled calibration set. However, so far it has been only discussed and empirically …
An efficient dataset condensation plugin and its application to continual learning
Dataset condensation (DC) distills a large real-world dataset into a small synthetic dataset,
with the goal of training a network from scratch on the latter that performs similarly to the …
with the goal of training a network from scratch on the latter that performs similarly to the …
Lq-lora: Low-rank plus quantized matrix decomposition for efficient language model finetuning
We propose a simple approach for memory-efficient adaptation of pretrained language
models. Our approach uses an iterative algorithm to decompose each pretrained matrix into …
models. Our approach uses an iterative algorithm to decompose each pretrained matrix into …
Complete dictionary recovery over the sphere II: Recovery by Riemannian trust-region method
We consider the problem of recovering a complete (ie, square and invertible) matrix A 0,
from Y∈ R n× p with Y= A 0 X 0, provided X 0 is sufficiently sparse. This recovery problem is …
from Y∈ R n× p with Y= A 0 X 0, provided X 0 is sufficiently sparse. This recovery problem is …
Weighted low rank approximations with provable guarantees
The classical low rank approximation problem is: given a matrix A, find a rank-k matrix B
such that the Frobenius norm of A− B is minimized. It can be solved efficiently using, for …
such that the Frobenius norm of A− B is minimized. It can be solved efficiently using, for …
Low rank matrix completion via robust alternating minimization in nearly linear time
Given a matrix $ M\in\mathbb {R}^{m\times n} $, the low rank matrix completion problem
asks us to find a rank-$ k $ approximation of $ M $ as $ UV^\top $ for $ U\in\mathbb …
asks us to find a rank-$ k $ approximation of $ M $ as $ UV^\top $ for $ U\in\mathbb …
Nonconvex Rectangular Matrix Completion via Gradient Descent Without ℓ₂,∞ Regularization
The analysis of nonconvex matrix completion has recently attracted much attention in the
community of machine learning thanks to its computational convenience. Existing analysis …
community of machine learning thanks to its computational convenience. Existing analysis …