Copy suppression: Comprehensively understanding an attention head
We present a single attention head in GPT-2 Small that has one main role across the entire
training distribution. If components in earlier layers predict a certain token, and this token …
training distribution. If components in earlier layers predict a certain token, and this token …
Successor heads: Recurring, interpretable attention heads in the wild
In this work we present successor heads: attention heads that increment tokens with a
natural ordering, such as numbers, months, and days. For example, successor heads …
natural ordering, such as numbers, months, and days. For example, successor heads …
Attribution patching outperforms automated circuit discovery
Automated interpretability research has recently attracted attention as a potential research
direction that could scale explanations of neural network behavior to large models. Existing …
direction that could scale explanations of neural network behavior to large models. Existing …
Information flow routes: Automatically interpreting language models at scale
J Ferrando, E Voita - arXiv preprint arXiv:2403.00824, 2024 - arxiv.org
Information flows by routes inside the network via mechanisms implemented in the model.
These routes can be represented as graphs where nodes correspond to token …
These routes can be represented as graphs where nodes correspond to token …
Dictionary learning improves patch-free circuit discovery in mechanistic interpretability: A case study on othello-gpt
Sparse dictionary learning has been a rapidly growing technique in mechanistic
interpretability to attack superposition and extract more human-understandable features …
interpretability to attack superposition and extract more human-understandable features …
Transformer circuit evaluation metrics are not robust
J Miller, B Chughtai, W Saunders - First Conference on Language …, 2024 - openreview.net
Mechanistic interpretability work attempts to reverse engineer the learned algorithms
present inside neural networks. One focus of this work has been to discover'circuits' …
present inside neural networks. One focus of this work has been to discover'circuits' …
Automatically Identifying and Interpreting Sparse Circuits with Hierarchical Tracing
We present a novel approach to Transformer circuit analysis using Sparse Autoencoders
(SAEs) and Transcoders. SAEs allow fine-grained feature extraction from model activations …
(SAEs) and Transcoders. SAEs allow fine-grained feature extraction from model activations …