This source only provides a headline — there is no body to show. Use “Read the original” below to read it at the source.
Tracing Feature Circuits
Neuronal superposition is the phenomena where a neuron in a model activates on many unrelated concepts. This is a problem for interpretability research because you don't have a 1:1 mapping of a neuron and a clean, human-interpretable feature. Sparse dictionary learning offers a potential solution to this problem whereby a neuron in a model is mapped, through an autoencoder, to each of the features it activates. It offers a method to distill the many features embedded in a single neuron to a single feature activation. Now imagine you could decompose each component of a model (the MLP layers and attentional units) into a sparse dictionary of features such that for a single input prompt, you have a mapping of which native model components correspond to which sparse dictionary features. The dream scenario is each of the autoencoders' features are monosemantic, interpretable, and you could replace each component (i.e. layer 5 attention head 6) with these features; thereby decomposing a blac…