Optimal embedding dimension for sparse subspace embeddings

S Chenakkod, M Dereziński, X Dong… - Proceedings of the 56th …, 2024 - dl.acm.org
A random m× n matrix S is an oblivious subspace embedding (OSE) with parameters є> 0,
δ∈(0, 1/3) and d≤ m≤ n, if for any d-dimensional subspace W⊆ R n, P (∀ x∈ W (1+ є)− 1 …

Solving dense linear systems faster than via preconditioning

M Dereziński, J Yang - Proceedings of the 56th Annual ACM Symposium …, 2024 - dl.acm.org
We give a stochastic optimization algorithm that solves a dense n× n real-valued linear
system Ax= b, returning x such that|| A x− b||≤ є|| b|| in time: Õ ((n 2+ nk ω− 1) log1/є), where …

Asymptotics of the sketched pseudoinverse

D LeJeune, P Patil, H Javadi, RG Baraniuk… - SIAM Journal on …, 2024 - SIAM
We take a random matrix theory approach to random sketching and show an asymptotic first-
order equivalence of the regularized sketched pseudoinverse of a positive semidefinite …

Mathematical models of computation in superposition

K Hänni, J Mendel, D Vaintrob, L Chan - arXiv preprint arXiv:2408.05451, 2024 - arxiv.org
Superposition--when a neural network represents more``features''than it has dimensions--
seems to pose a serious challenge to mechanistically interpreting current AI systems …

Randomized sketching of nonlinear eigenvalue problems

S Güttel, D Kressner, B Vandereycken - SIAM Journal on Scientific Computing, 2024 - SIAM
Rational approximation is a powerful tool to obtain accurate surrogates for nonlinear
functions that are easy to evaluate and linearize. The interpolatory adaptive Antoulas …

Hermitian dynamic mode decomposition-numerical analysis and software solution

Z Drmač - ACM Transactions on Mathematical Software, 2024 - dl.acm.org
The Dynamic Mode Decomposition (DMD) is a versatile and increasingly popular method for
data driven analysis of dynamical systems that arise in a variety of applications in, eg …

Asymptotically free sketched ridge ensembles: Risks, cross-validation, and tuning

P Patil, D LeJeune - arXiv preprint arXiv:2310.04357, 2023 - arxiv.org
We employ random matrix theory to establish consistency of generalized cross validation
(GCV) for estimating prediction risks of sketched ridge regression ensembles, enabling …

Generalizing random butterfly transforms to arbitrary matrix sizes

N Lindquist, P Luszczek, J Dongarra - ACM Transactions on …, 2023 - dl.acm.org
Parker and Lê introduced random butterfly transforms (RBTs) as a preprocessing technique
to replace pivoting in dense LU factorization. Unfortunately, their FFT-like recursive structure …

A LAPACK implementation of the Dynamic Mode Decomposition

Z Drmač - ACM Transactions on Mathematical Software, 2022 - dl.acm.org
The Dynamic Mode Decomposition (DMD) is a method for computational analysis of
nonlinear dynamical systems in data driven scenarios. Based on high fidelity numerical …

Constrained optimization via exact augmented lagrangian and randomized iterative sketching

I Hong, S Na, MW Mahoney… - … Conference on Machine …, 2023 - proceedings.mlr.press
We consider solving equality-constrained nonlinear, nonconvex optimization problems. This
class of problems appears widely in a variety of applications in machine learning and …