Interactive references and an annotated tech report — from DeepSeek LLM to V4.1-Flash
A working set of notes on the DeepSeek model family. Start with the papers map for the big picture, then use the three interactive pages to see how the V4.1-Flash architecture actually moves tokens, keys, and weights around. The commented tech report is the primary source behind all of it.
PDF · 2.3 MB · annotated copy
The V4.1-Flash technical report, Pushing the Limits of KV Cache Compression, with my margin notes and highlights. The comments call out what each design choice buys, where the numbers come from, and which sections the interactive pages below are built from (mainly §2 Architecture and §3 Infrastructures, Figures 3 to 5).
Open PDFInteractive · 2024 to 2026 · 16 papers and releases
A clickable timeline of the DeepSeek line of work, from the original DeepSeek LLM scaling paper through DeepSeekMoE, V2, V3, R1, V3.2, mHC, Engram, V4 and V4.1-Flash, plus the external YOCO paper that V4 builds on. Switch to the Themes view to see the same papers arranged by the idea they advance: scaling and data, sparse MoE, attention and KV cache, reasoning via RL, long context and memory, residual topology, and domains and modality. Click any block for a short summary and a link to the paper.
Open mapInteractive reference · SWA, NSA, DSA, CSA, CSA2, HCA
Why attention is expensive, and every trick used to avoid paying the full quadratic price. Starts with sliding window attention, lays out a taxonomy of sparse patterns (fixed, dynamic, natively trained), then goes deep on Compressed Sparse Attention and its CSA2 successor as used in V4.1-Flash. Hover the attention-mask grids to see what a single query reads. Ends with a GPU cost calculator that gives floating-point operations and bytes moved per decode step for each scheme.
Open walkthroughAnimated diagram · all 40 layers unrolled
The full V4.1-Flash stack drawn layer by layer: the 20-layer causal encoder, the 20-layer decoder, which layers compute their own KV and which reuse another layer's, where the Top-K indices come from, and where the two Engram layers sit. Five scrubbable animations show the model in different regimes:
Click any layer for its own panel. The scrubber is authoritative, so you can drag to any frame.
Open anatomyInteractive schematic · dataflow across the hardware hierarchy
A deliberately small 5-plus-5 layer schematic of encoder and decoder, placed on top of the memory hierarchy it runs on: GPU SRAM, HBM, host DRAM, and NVMe SSD, with width for capacity and stripe for latency. The point is to separate things that are easy to conflate: the compute path, the shared KV cache, the SWA local window, FP4 stored values, and Engram tables, which are weights rather than activations. Animate prefill, next-token generation, an SWA cache miss, or an Engram lookup, and toggle CSA2 layer policies (Full, Reindex, Reuse) and FP4 global KV to see what moves and what it costs.
Open explorer