Interactive references and an annotated tech report — from DeepSeek LLM to V4.1-Flash
A working set of notes on the DeepSeek model family. Start with the papers map for the big picture, then use the three interactive pages to see how the V4.1-Flash architecture actually moves tokens, keys, and weights around. The commented tech report is the primary source behind all of it.
PDF · 2.3 MB · annotated copy
The V4.1-Flash technical report, Pushing the Limits of KV Cache Compression, with my margin notes and highlights. The comments call out what each design choice buys, where the numbers come from, and which sections the interactive pages below are built from (mainly §2 Architecture and §3 Infrastructures, Figures 3 to 5).
Open PDFInteractive · 2024 to 2026 · 16 papers and releases
A clickable timeline of the DeepSeek line of work, from the original DeepSeek LLM scaling paper through DeepSeekMoE, V2, V3, R1, V3.2, mHC, Engram, V4 and V4.1-Flash, plus the external YOCO paper that V4 builds on. Switch to the Themes view to see the same papers arranged by the idea they advance: scaling and data, sparse MoE, attention and KV cache, reasoning via RL, long context and memory, residual topology, and domains and modality. Click any block for a short summary and a link to the paper.
Open mapInteractive reference · SWA, NSA, DSA, CSA, CSA2, HCA
Why attention is expensive, and every trick used to avoid paying the full quadratic price. Starts with sliding window attention, lays out a taxonomy of sparse patterns (fixed, dynamic, natively trained), then goes deep on Compressed Sparse Attention and its CSA2 successor as used in V4.1-Flash. Hover the attention-mask grids to see what a single query reads. Ends with a GPU cost calculator that gives floating-point operations and bytes moved per decode step for each scheme.
Open walkthroughAnimated diagram · all 40 layers unrolled
The full V4.1-Flash stack drawn layer by layer: the 20-layer causal encoder, the 20-layer decoder, which layers compute their own KV and which reuse another layer's, where the Top-K indices come from, and where the two Engram layers sit. Five scrubbable animations show the model in different regimes:
Click any layer for its own panel. The scrubber is authoritative, so you can drag to any frame.
Open anatomyInteractive schematic · dataflow across the hardware hierarchy
A deliberately small 5-plus-5 layer schematic of encoder and decoder, placed on top of the memory hierarchy it runs on: GPU SRAM, HBM, host DRAM, and NVMe SSD, with width for capacity and stripe for latency. The point is to separate things that are easy to conflate: the compute path, the shared KV cache, the SWA local window, FP4 stored values, and Engram tables, which are weights rather than activations. Animate prefill, next-token generation, an SWA cache miss, or an Engram lookup, and toggle CSA2 layer policies (Full, Reindex, Reuse) and FP4 global KV to see what moves and what it costs.
Open explorerInteractive diagram · Tech Report §4 to §5 · every box clickable
Pre-training: raw sources (a re-bootstrapped web crawl, curated text and fresh code, interleaved webpages and PDFs, domain-specific data) flowing through the implicit-duplication filter, dedup and quality models, the five-stage multimodal pipeline, and corpus integration into the 45-trillion-token run that produces the base model.
Post-training: Reinforcement learning loop. A task factory feeds the asynchronous RL loop on the DSec substrate. Four animated flows.
Click any box for its explanation. Zoom controls and a resizable panel are in the toolbar.
Open pipelinePDF · arXiv 2609.22978 · annotated copy
The paper describing DSec, the elastic sandbox platform that runs the environments and rollouts for V4.1-Flash post-training. It covers the stateless control plane, the four isolation backends, the 3FS storage layer, and the mechanisms for bursting rollouts, loading environments on demand, and preempting and resuming work. My margin notes tie the section numbers to the boxes in the animated diagram below.
Open PDFInteractive diagram · paper Figure 1 redrawn · every box clickable
The DSec architecture on one canvas. Clients on the left (trainer, RL framework, the libdsec SDK and pack_diff environments), stateless cluster services and a single worker node in the middle (IAM, API server, placement engine, watcher), the §5 mechanisms on the right, and 3FS storage along the bottom. Four views animate one story at a time:
Each box cites the paper section it comes from and links into the annotated PDF.
Open anatomy