AI is undergoing a paradigm shift with the rise of models (eg, BERT, DALL-E, GPT-3) that are
trained on broad data at scale and are adaptable to a wide range of downstream tasks. We …

The rise and success of large language models undermines virtually every strong claim for
the innateness of language that has been proposed by generative linguistics. Modern …

A possible explanation for the impressive performance of masked language model (MLM)
pre-training is that such models have learned to represent the syntactic structures prevalent …

Rapid progress in machine learning for natural language processing has the potential to
transform debates about how humans learn language. However, the learning environments …

What kind of knowledge accounts for linguistic productivity? How is it acquired? For years,
debate on these questions has focused on a seemingly obscure domain: inflectional …

Recent studies have shown that language models pretrained and/or fine-tuned on randomly
permuted sentences exhibit competitive performance on GLUE, putting into question the …

A central quest of probing is to uncover how pre-trained models encode a linguistic property
within their representations. An encoding, however, might be spurious-ie, the model might …

Recent studies show that pre-trained language models (LMs) are vulnerable to textual
adversarial attacks. However, existing attack methods either suffer from low attack success …

We investigate what kind of structural knowledge learned in neural network encoders is
transferable to processing natural language. We design artificial languages with structural …

Because meaning can often be inferred from lexical semantics alone, word order is often a
redundant cue in natural language. For example, the words chopped, chef, and onion are …