Multiplicative filter networks

R Fathony, AK Sahu, D Willmott… - … Conference on Learning …, 2020 - openreview.net
Although deep networks are typically used to approximate functions over high dimensional
inputs, recent work has increased interest in neural networks as function approximators for …

Random features for kernel approximation: A survey on algorithms, theory, and beyond

F Liu, X Huang, Y Chen… - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
The class of random features is one of the most popular techniques to speed up kernel
methods in large-scale problems. Related works have been recognized by the NeurIPS Test …

An improved cutting plane method for convex optimization, convex-concave games, and its applications

H Jiang, YT Lee, Z Song, SC Wong - … of the 52nd Annual ACM SIGACT …, 2020 - dl.acm.org
Given a separation oracle for a convex set K⊂ ℝ n that is contained in a box of radius R, the
goal is to either compute a point in K or prove that K does not contain a ball of radius є. We …

A random matrix analysis of random fourier features: beyond the gaussian kernel, a precise phase transition, and the corresponding double descent

Z Liao, R Couillet, MW Mahoney - Advances in Neural …, 2020 - proceedings.neurips.cc
This article characterizes the exact asymptotics of random Fourier feature (RFF) regression,
in the realistic setting where the number of data samples $ n $, their dimension $ p $, and …

Towards a unified analysis of random Fourier features

Z Li, JF Ton, D Oglic… - … conference on machine …, 2019 - proceedings.mlr.press
Random Fourier features is a widely used, simple, and effective technique for scaling up
kernel methods. The existing theoretical analysis of the approach, however, remains …

Oblivious sketching of high-degree polynomial kernels

TD Ahle, M Kapralov, JBT Knudsen, R Pagh… - Proceedings of the …, 2020 - SIAM
Kernel methods are fundamental tools in machine learning that allow detection of non-linear
dependencies between data without explicitly constructing feature vectors in high …

Fast sketching of polynomial kernels of polynomial degree

Z Song, D Woodruff, Z Yu… - … Conference on Machine …, 2021 - proceedings.mlr.press
Kernel methods are fundamental in machine learning, and faster algorithms for kernel
approximation provide direct speedups for many core tasks in machine learning. The …

Training (overparametrized) neural networks in near-linear time

J Brand, B Peng, Z Song, O Weinstein - arXiv preprint arXiv:2006.11648, 2020 - arxiv.org
The slow convergence rate and pathological curvature issues of first-order gradient methods
for training deep neural networks, initiated an ongoing effort for developing faster $\mathit …

Memorization and optimization in deep neural networks with minimum over-parameterization

S Bombari, MH Amani… - Advances in Neural …, 2022 - proceedings.neurips.cc
Abstract The Neural Tangent Kernel (NTK) has emerged as a powerful tool to provide
memorization, optimization and generalization guarantees in deep neural networks. A line of …

Quasi-monte carlo graph random features

I Reid, A Weller… - Advances in Neural …, 2024 - proceedings.neurips.cc
We present a novel mechanism to improve the accuracy of the recently-introduced class of
graph random features (GRFs). Our method induces negative correlations between the …