Ew-tune: A framework for privately fine-tuning large language models with differential privacy
Pre-trained Large Language Models (LLMs) are an integral part of modern AI that have led
to breakthrough performances in complex AI tasks. Major AI companies with expensive …
to breakthrough performances in complex AI tasks. Major AI companies with expensive …
Differentially private fine-tuning of language models
We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-
scale pre-trained language models, which achieve the state-of-the-art privacy versus utility …
scale pre-trained language models, which achieve the state-of-the-art privacy versus utility …
Large language models can be good privacy protection learners
The proliferation of Large Language Models (LLMs) has driven considerable interest in fine-
tuning them with domain-specific data to create specialized language models. Nevertheless …
tuning them with domain-specific data to create specialized language models. Nevertheless …
Dp-forward: Fine-tuning and inference on language models with differential privacy in forward pass
Differentially private stochastic gradient descent (DP-SGD) adds noise to gradients in back-
propagation, safeguarding training data from privacy leakage, particularly membership …
propagation, safeguarding training data from privacy leakage, particularly membership …
Differentially private decoding in large language models
Recent large-scale natural language processing (NLP) systems use a pre-trained Large
Language Model (LLM) on massive and diverse corpora as a headstart. In practice, the pre …
Language Model (LLM) on massive and diverse corpora as a headstart. In practice, the pre …
Privacy-preserving prompt tuning for large language model services
Prompt tuning provides an efficient way for users to customize Large Language Models
(LLMs) with their private data in the emerging LLM service scenario. However, the sensitive …
(LLMs) with their private data in the emerging LLM service scenario. However, the sensitive …
Learning differentially private recurrent language models
We demonstrate that it is possible to train large recurrent language models with user-level
differential privacy guarantees with only a negligible cost in predictive accuracy. Our work …
differential privacy guarantees with only a negligible cost in predictive accuracy. Our work …
Differentially private language models benefit from public pre-training
Language modeling is a keystone task in natural language processing. When training a
language model on sensitive information, differential privacy (DP) allows us to quantify the …
language model on sensitive information, differential privacy (DP) allows us to quantify the …
Natural language understanding with privacy-preserving bert
Privacy preservation remains a key challenge in data mining and Natural Language
Understanding (NLU). Previous research shows that the input text or even text embeddings …
Understanding (NLU). Previous research shows that the input text or even text embeddings …
Privacy preserving large language models: Chatgpt case study based vision and framework
The generative Artificial Intelligence (AI) tools based on Large Language Models (LLMs) use
billions of parameters to extensively analyse large datasets and extract critical private …
billions of parameters to extensively analyse large datasets and extract critical private …