Significant theoretical work has established that in specific regimes, neural networks trained
by gradient descent behave like kernel methods. However, in practice, it is known that …

In suitably initialized wide networks, small learning rates transform deep neural networks
(DNNs) into neural tangent kernel (NTK) machines, whose training dynamics is well …

The combination of optical microresonators and the emerging microwave photonic (MWP)
sensing has recently drawn great attention, whereas its multi-parameter sensing capability …

Abstract Neural Tangent Kernel (NTK) is widely used to analyze overparametrized neural
networks due to the famous result by Jacot et al.(2018): in the infinite-width limit, the NTK is …

Offline model-based optimization aims to maximize a black-box objective function with a
static dataset of designs and their scores. In this paper, we focus on biological sequence …

We study the multiple manifold problem, a binary classification task modeled on applications
in machine vision, in which a deep fully-connected neural network is trained to separate two …

Deep neural networks can empirically perform efficient hierarchical learning, in which the
layers learn useful representations of the data. However, how they make use of the …

Classifiers that are linear in their parameters, and trained by optimizing a convex loss
function, have predictable behavior with respect to changes in the training data, initial …

In this paper we study the training dynamics for gradient flow on over-parametrized tensor
decomposition problems. Empirically, such training process often first fits larger components …

It is often useful to compactly summarize important properties of model parameters and
training data so that they can be used later without storing and/or iterating over the entire …