AI alignment aims to make AI systems behave in line with human intentions and values. As
AI systems grow more capable, the potential large-scale risks associated with misaligned AI …

With the development of large language models (LLMs), striking a balance between the
performance and safety of AI systems has never been more critical. However, the inherent …

Safety lies at the core of the development of Large Language Models (LLMs). There is
ample work on aligning LLMs with human ethics and preferences, including data filtering in …

This survey presents an in-depth exploration of knowledge distillation (KD) techniques
within the realm of Large Language Models (LLMs), spotlighting the pivotal role of KD in …

The widespread application of Reinforcement Learning (RL) in real-world situations is yet to
come to fruition, largely as a result of its failure to satisfy the essential safety demands of …

Preference datasets are essential for incorporating human preferences into pre-trained
language models, playing a key role in the success of Reinforcement Learning from Human …

The current safeguard mechanisms for large language models (LLMs) are indeed
susceptible to jailbreak attacks, making them inherently fragile. Even the process of fine …

Recent advances in large language models have shown capabilities that are extraordinary
and near-superhuman. These models operate with such complexity that reliably evaluating …

In this work, we introduce the PKU-SafeRLHF dataset, designed to promote research on
safety alignment in large language models (LLMs). As a sibling project to SafeRLHF and …

Although Large Language Models (LLMs) are becoming increasingly powerful, they still
exhibit significant but subtle weaknesses, such as mistakes in instruction-following or coding …