As large language models (LLMs) become easily accessible nowadays, the trade-off
between safety and helpfulness can significantly impact user experience. A model that …

Fine-tuning large language models (LLMs) on human preferences, typically through
reinforcement learning from human feedback (RLHF), has proven successful in enhancing …

Recent progress in large language models (LLMs) has led to their widespread adoption in
various domains. However, these advancements have also introduced additional safety …

Reinforcement learning based fine-tuning of large language models (LLMs) on human
preferences has been shown to enhance both their capabilities and safety behavior …

As large language models (LLMs) become increasingly integrated into daily applications, it
is essential to ensure they operate fairly across diverse user demographics. In this work, we …

Safety alignment is the key to guiding the behaviors of large language models (LLMs) that
are in line with human preferences and restrict harmful behaviors at inference time, but …

In this work, we introduce the PKU-SafeRLHF dataset, designed to promote research on
safety alignment in large language models (LLMs). As a sibling project to SafeRLHF and …

Frontier Large Language Models (LLMs) are increasingly being deployed for high-stakes
decision-making. On the other hand, these models are still consistently making predictions …

Large language models (LLMs) have shown great potential as general-purpose AI
assistants in various domains. To meet the requirements of different applications, LLMs are …

In this paper, we introduce the BeaverTails dataset, aimed at fostering research on safety
alignment in large language models (LLMs). This dataset uniquely separates annotations of …