Probing classifiers have emerged as one of the prominent methodologies for interpreting
and analyzing deep neural network models of natural language processing. The basic idea …

This work identifies 18 foundational challenges in assuring the alignment and safety of large
language models (LLMs). These challenges are organized into three different categories …

Petroni et al.(2019) demonstrated that it is possible to retrieve world facts from a pre-trained
language model by expressing them as cloze-style prompts and interpret the model's …

摘要随着大规模预训练模型的广泛应用, 自然语言处理的多个领域(如文本分类和机器翻译)
取得了长足的发展. 然而, 受限于预训练模型的“黑盒” 特性, 其内部的决策模式以及编码的知识 …

Large language models have achieved success on a number of downstream tasks,
particularly in a few and zero-shot manner. As a consequence, researchers have been …

Probing strategies have been shown to detect the presence of various linguistic features in
large language models; in particular, semantic features intermediate to the" natural logic" …

Several studies have been carried out on revealing linguistic features captured by BERT.
This is usually achieved by training a diagnostic classifier on the representations obtained …

Neural models achieve high performance on a variety of natural language processing (NLP)
benchmark tasks. How models perform these tasks, though, is notoriously poorly …

Large NLP models have recently shown impressive performance in language
understanding tasks, typically evaluated by their fine-tuned performance. Alternatively …

Pixel-based language models have emerged as a compelling alternative to subword-based
language modelling, particularly because they can represent virtually any script. PIXEL, a …