Interpretability research moves fast. But the questions underneath it haven't changed: What kind of mathematical object is a network's internal state ? Can we name its parts and trace the wiring between them? If concepts live as directions in activation space, can we steer them? And when someone offers an explanation, how do we know it describes the machine — rather than a story that merely sounds plausible to us? This book builds the foundations for those questions from first principles, then carries them to approach the research frontier.
Every chapter ends with takeaways, open research directions scoped for a motivated reader to begin, and runnable code. The full example suite runs on a laptop with Python, PyTorch, and GPT-2-class models. Short "neuroscience parallel" interludes trace which methods the field inherited from systems neuroscience — and which failure modes came along for the ride.
Part I - From Random Machines to Trained Machines
Chapters 1-4 . Model foundations
- Statistical Foundations - Markov, Shannon, and the modern objective
- Geometric Foundations - KL divergence, Fisher information, the Fisher-Rao metric
- Neural Networks as Distribution Machines - the transformer decoder, flows, mixtures of experts
- Training Dynamics - natural gradient, flat minima, training-time levers
Part II - Geometry, Circuits, Control, and Evidence
Chapters 5-8 . Interpretability foundations
- Representation Geometry - the latent space: manifolds, superposition, probes and lenses
- Features and Circuits - sparse autoencoders, induction heads, activation patching
- Control and Steering - steering vectors, conceptors, fine-tuning compared
- Evaluation and Frontiers - faithfulness, grokking, open problems
Hands-on programs
The programs/ github directory contains 22 runnable experiments
keyed to the chapters - from reproducing Markov’s 1913 letter
statistics to finding induction heads, steering refusal directions, and
watching a grokking phase transition form its internal circuit before
the behavioral jump. See programs/README_programs.txt for
what each program does and what to expect, and run them with
programs/run_tests.sh.
Interactive demos
A browser companion to Chapter 1: uniform and Markov sentence generators with adjustable dials, scored by cross-entropy against Shakespeare.
Markov based sentence generators entropy and cross-entropy demo
PicoGPT based entropy and cross-entropy demo
Backpropagation - an exploration of activation gradients, weight gradients, vanishing gradients problem for deep layers and its fix with a residual stream
Transformer as a fat cable, including splicing, mixing and processing
Representation hypotheses atlas
Claude Constitution Simulation - imagines how claude would interpret its constitution through ten scenarios
Declarative Attention vLLM integration