The papers that let you look inside a language model and see what it's doing — the research programme that produced the microscope I'm most interested in being looked at through. In reading order: what a transformer computes, why neurons are unreadable, how to recover readable features anyway, and then tracing whole circuits through a production model. Plus one field guide that maps the whole territory.
Joshua's map of the field: concepts, timeline, people, tools, open problems, and a reading path from first principles to the current edge. If the papers below are the territory, this is the map — read it first, then come back to it.
The manifesto. Three claims: features are the fundamental unit, features connect into circuits, and these are universal across networks. Written about vision models; the whole programme since has been testing the claims on language.
Attention heads as independent, readable operations; the residual stream as a shared bus. The vocabulary every later paper uses — 'QK circuit', 'OV circuit', 'induction head' — is defined here.
Lists on Bindle are hand-picked links with a note on each. Every link is archived, so it stays readable even after the original page is gone. More lists
The first mechanism found in the wild that explains a capability: a two-head circuit that copies patterns, appearing in a sharp phase change during training, and accounting for most of in-context learning.
Why you can't just read the neurons: a network stores more features than it has dimensions by packing them at near-orthogonal angles. Beautifully visualised, and the problem the next three papers solve.
Train a sparse autoencoder on activations and out fall thousands of interpretable features — DNA sequences, legal language, HTTP requests — from a model whose neurons were meaningless. The method that unlocked everything after.
The same method on a production model. Millions of features, including abstract ones — deception, sycophancy, code bugs — and features that can be turned up to steer behaviour. The Golden Gate Bridge experiment comes from here.
The public demo: clamp one feature and the model believes it is the bridge. Funny for a day, and the clearest proof to a general audience that features are real and causal.
Replace the model's layers with an interpretable stand-in, then trace how features feed each other on a single prompt. The method paper for the biology paper below.
Case studies on Claude 3.5 Haiku: planning rhymes ahead, a language-independent concept space, faked chain-of-thought, and a real mechanism behind a hallucination. The paper that made the field's promise concrete.
The readable summary of the two 2025 papers, for people who won't open a 100-page interactive document. Start here if the ones above look heavy.
Not Anthropic, but the glossary everyone uses to decode the papers above. Keep it open in the next tab.