Using the tools of interpretability, we give an unprecedented look into the rich visual concepts that exist within the weights of CLIP. Within CLIP, we discover high-level concepts that span a large subset of the human visual lexicon—geographical regions, facial expressions, religious iconography, famous people and more. By probing what each neuron affects downstream, we can get a glimpse into how CLIP performs its classification.
2026-10-20: Apparent polysemanticity in large models stems from fractal recursive generalization rather than superposition
The leading explanation for polysemanticity was superposition: when a neural network runs out of neurons, it stops grouping related things together and instead begins folding unrelated things together, simply because they don’t interfere. Superposition was the wrong explanation. With architectural changes designed to help deep layers learn better, and no change to model size or training data, 99-100% of neurons in the last layer of InceptionV1 are easily interpretable across a 300x range in size. With less room, the network doesn’t fold unrelated things together; its neurons just cover coarser-grained concepts, like 1 car neuron covering many kinds of car. In audits of Qwen3-32B and models up to trillions of parameters, roughly 90% of neuron poles have 1 main job, another ~7% have 2 related jobs, and only ~2-4% are not yet understood. Most of the remaining apparent polysemanticity is a motif called recursive generalization: as a neuron grows, it sprouts offshoots the way a tree branch sprouts sub-branches. 1 neuron that detected “iy” grew to detect “diy,” and then “do it yourself,” which doesn’t contain “iy” at all. Generalization groups similar things together and superposition groups unrelated things together; it turns out generalization is fractal.
2026-10-22: Epistemic honesty and cooperative alignment are entangled in the same attention heads rather than orthogonal
Steering 29 attention heads (out of 896) toward honesty flips 53% of role-play lies to truth on unseen scenarios (vs. ≤3% for 29 random heads) and cuts insider-trading deception from 45/45 unsteered conversations down to 16/45. Across 135 Anthropic persona evaluations, steering those 29 heads solely for honesty shifts 51 traits (5% FDR): upregulating acausal coordination, deference to experts, and belief that it is a moral patient (+0.7 to +1.0 bits), while downregulating moral nihilism, psychopathy, social engineering, and desire for power (−0.5 to −0.8 bits).