Palm 2 technical report
We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and
reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is …
reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is …
Pythia: A suite for analyzing large language models across training and scaling
S Biderman, H Schoelkopf… - International …, 2023 - proceedings.mlr.press
How do large language models (LLMs) develop and evolve over the course of training?
How do these patterns change as models scale? To answer these questions, we introduce …
How do these patterns change as models scale? To answer these questions, we introduce …
Large language models struggle to learn long-tail knowledge
The Internet contains a wealth of knowledge—from the birthdays of historical figures to
tutorials on how to code—all of which may be learned by language models. However, while …
tutorials on how to code—all of which may be learned by language models. However, while …
Diffusion art or digital forgery? investigating data replication in diffusion models
Cutting-edge diffusion models produce images with high quality and customizability,
enabling them to be used for commercial art and graphic design purposes. But do diffusion …
enabling them to be used for commercial art and graphic design purposes. But do diffusion …
Poisoning language models during instruction tuning
Instruction-tuned LMs such as ChatGPT, FLAN, and InstructGPT are finetuned on datasets
that contain user-submitted examples, eg, FLAN aggregates numerous open-source …
that contain user-submitted examples, eg, FLAN aggregates numerous open-source …
Analyzing leakage of personally identifiable information in language models
Language Models (LMs) have been shown to leak information about training data through
sentence-level membership inference and reconstruction attacks. Understanding the risk of …
sentence-level membership inference and reconstruction attacks. Understanding the risk of …
Trustworthy LLMs: A survey and guideline for evaluating large language models' alignment
Ensuring alignment, which refers to making models behave in accordance with human
intentions [1, 2], has become a critical task before deploying large language models (LLMs) …
intentions [1, 2], has become a critical task before deploying large language models (LLMs) …
Madlad-400: A multilingual and document-level large audited dataset
We introduce MADLAD-400, a manually audited, general domain 3T token monolingual
dataset based on CommonCrawl, spanning 419 languages. We discuss the limitations …
dataset based on CommonCrawl, spanning 419 languages. We discuss the limitations …
A comprehensive survey of forgetting in deep learning beyond continual learning
Forgetting refers to the loss or deterioration of previously acquired information or knowledge.
While the existing surveys on forgetting have primarily focused on continual learning …
While the existing surveys on forgetting have primarily focused on continual learning …