Abstract Machine Learning (ML) models are ubiquitous in real-world applications and are a
constant focus of research. Modern ML models have become more complex, deeper, and …

Differentially Private (DP) learning has seen limited success for building large deep learning
models of text, and straightforward attempts at applying Differentially Private Stochastic …

We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-
scale pre-trained language models, which achieve the state-of-the-art privacy versus utility …

Large language models (LLMs) are excellent in-context learners. However, the sensitivity of
data contained in prompts raises privacy concerns. Our work first shows that these concerns …

Natural language reflects our private lives and identities, making its privacy concerns as
broad as those of real life. Language models lack the ability to understand the context and …

Large language models (LLMs) are complex artificial intelligence systems capable of
understanding, generating and translating human language. They learn language patterns …

Are Large Pre-Trained Language Models Leaking Your Personal Information? In this paper,
we analyze whether Pre-Trained Language Models (PLMs) are prone to leaking personal …

Recent developments in deep learning have led to great success in various natural
language processing (NLP) tasks. However, these applications may involve data that …

In this work, we study the large-scale pretraining of BERT-Large with differentially private
SGD (DP-SGD). We show that combined with a careful implementation, scaling up the batch …

Prompt tuning provides an efficient way for users to customize Large Language Models
(LLMs) with their private data in the emerging LLM service scenario. However, the sensitive …