We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-
scale pre-trained language models, which achieve the state-of-the-art privacy versus utility …

In the privacy-utility tradeoff of a model trained on benchmark language and vision tasks,
remarkable improvements have been widely reported when the model is pretrained on …

We provide an information-theoretic framework for studying the generalization properties of
machine learning algorithms. Our framework ties together existing approaches, including …

The privacy leakage of the model about the training data can be bounded in the differential
privacy mechanism. However, for meaningful privacy parameters, a differentially private …

We introduce AdaMix, an adaptive differentially private algorithm for training deep neural
network classifiers using both private and public image data. While pre-training language …

Differentially private SGD (DP-SGD) is one of the most popular methods for solving
differentially private empirical risk minimization (ERM). Due to its noisy perturbation on each …

In this paper, we revisit the problem of using in-distribution public data to improve the
privacy/utility trade-offs for differentially private (DP) model training.(Here, public data refers …

We study the problem of private distribution learning with access to public data. In this setup,
which we refer to as* public-private learning*, the learner is given public and private …

We initiate the study of differentially private (DP) estimation with access to a small amount of
public data. For private estimation of $ d $-dimensional Gaussians, we assume that the …

In many statistical problems, incorporating priors can significantly improve performance.
However, the use of prior knowledge in differentially private query release has remained …