The logit outputs of a feedforward neural network at initialization are conditionally Gaussian,
given a random covariance matrix defined by the penultimate layer. In this work, we study …

Abstract Neural Tangent Kernel (NTK) is widely used to analyze overparametrized neural
networks due to the famous result by Jacot et al.(2018): in the infinite-width limit, the NTK is …

We show that taking the width and depth to infinity in a deep neural network with skip
connections, when branches are scaled by $1/\sqrt {depth} $, result in the same covariance …

Theoretical results show that neural networks can be approximated by Gaussian processes
in the infinite-width limit. However, for fully connected networks, it has been previously …

In a backdoor attack, an attacker injects corrupted examples into the training set. The goal of
the attacker is to cause the final trained model to predict the attacker's desired target label …

This work bridges two important concepts: the Neural Tangent Kernel (NTK), which captures
the evolution of deep neural networks (DNNs) during training, and the Neural Collapse (NC) …

In this paper, we study the infinite-depth limit of finite-width residual neural networks with
random Gaussian weights. With proper scaling, we show that by fixing the width and taking …

While significant theoretical progress has been achieved, unveiling the generalization
mystery of overparameterized neural networks still remains largely elusive. In this paper, we …

We revisit on-average algorithmic stability of Gradient Descent (GD) for training
overparameterised shallow neural networks and prove new generalisation and excess risk …

It is often useful to compactly summarize important properties of model parameters and
training data so that they can be used later without storing and/or iterating over the entire …