Influenced by the great success of deep learning via cloud computing and the rapid
development of edge chips, research in artificial intelligence (AI) has shifted to both of the …

This chapter provides approaches to the problem of quantizing the numerical values in deep
Neural Network computations, covering the advantages/disadvantages of current methods …

We propose a novel neural representation for videos (NeRV) which encodes videos in
neural networks. Unlike conventional representations that treat videos as frame sequences …

Nowadays, the training data and neural network models are getting increasingly large. The
training time of deep learning will become unbearably long on a single machine. To reduce …

Communication of model updates between client nodes and the central aggregating server
is a major bottleneck in federated learning, especially in bandwidth-limited settings and high …

Federated learning data is drawn from a distribution of distributions: clients are drawn from a
meta-distribution, and their data are drawn from local data distributions. Thus generalization …

Byzantine-robustness has been gaining a lot of attention due to the growth of the interest in
collaborative and federated learning. However, many fruitful directions, such as the usage of …

Gradient compression (eg, gradient quantization and gradient sparsification) is a core
technique in reducing communication costs in distributed learning systems. The recent trend …

In training of modern large natural language processing (NLP) models, it has become a
common practice to split models using 3D parallelism to multiple GPUs. Such technique …

We study the problem of locally private mean estimation of high-dimensional vectors in the
Euclidean ball. Existing algorithms for this problem either incur sub-optimal error or have …