Deep reinforcement learning (RL) algorithms are predominantly evaluated by comparing their relative performance on a large suite of tasks. Most published results on deep RL …
Universality is a key hypothesis in mechanistic interpretability–that different models learn similar features and circuits when trained on similar tasks. In this work, we study the …
Pretraining is the preliminary and fundamental step in developing capable language models (LM). Despite this, pretraining data design is critically under-documented and often guided …
J Geiping, T Goldstein - International Conference on …, 2023 - proceedings.mlr.press
Recent trends in language modeling have focused on increasing performance through scaling, and have resulted in an environment where training language models is out of …
We present a conceptual framework, datamodeling, for analyzing the behavior of a model class in terms of the training data. For any fixed" target" example $ x $, training set $ S $, and …
Understanding Transformer-based models has attracted significant attention, as they lie at the heart of recent technological advances across machine learning. While most …
Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of …
Similarity measures are a vital tool for understanding how language models represent and process language. Standard representational similarity measures such as cosine similarity …
Customer satisfaction is one of the most important measures in the hospitality industry. Therefore, several psychological and cognitive theories have been utilized to provide …