0:00For a year, the biggest mystery in AI wasn't how DeepSeek trained its models — it was how it sold access so cheaply. The common theory was that it was burning venture capital to undercut Western models. Hours ago, DeepSeek published the actual numbers from inside its data centers and open-sourced the software behind them: DeepInfer (the inference engine) and deepseek-recipe (the tokenizer/gateway). A year and a half after saying open-sourcing their serving stack was "impossible," they did it.
The staggering numbers
0:35In a single cluster, the system processes 155 trillion tokens per day — roughly 1.8 billion words per second, around the clock. Even at discounted off-peak pricing, a single GPU running their setup generates ~$38 of revenue per hour; against an enterprise chip costing roughly $5–10/hour to run, DeepSeek is operating at an estimated ~80% gross margin — not losing money at all. 1:27The repo hit ~9,000 stars in its first 24 hours and sits near 20,000 as developers dig in.
The architecture
1:38The core idea is a causal encoder-decoder design. A typical ~40-layer model forces your prompt through every layer before it can write the first word. DeepSeek splits that assembly line in half: the bottom 20 layers are a dedicated reader, the top 20 a dedicated writer. Your prompt only runs through the reader; the writer's work is projected mathematically from the bottom half, cutting the heavy reading work in half. 2:33On top, it's a mixture-of-experts: 748 billion total parameters, but only 8 billion wake up to read your prompt and 16 billion to write the answer.
Four engineering moves
2:471. Wide expert parallelism. Reading is compute-bound; writing (one word at a time) is memory-bound — every word forces the GPU to fetch billions of weights. DeepInfer splits them: ordinary 8-GPU servers for reading, and 64–128-GPU supernodes for writing. The 384 experts are spread across 128 chips (~7 each), cutting memory traffic ~88% and letting a single GPU emit 26,000+ words/second.
4:142. EPLB — expert parallelism load balancer. When thousands of coding queries hammer the "coding" experts while the creative-writing experts idle, the system waits on the slowest GPU. EPLB monitors traffic and dynamically spins up duplicate copies of overwhelmed experts on idle GPUs — a grocery store opening new checkout lanes the moment a line forms.
5:023. DSpark — speculative decoding with a "confidence head." A small drafter model guesses the next few words and the big model verifies them. The usual problem: if word three is wrong, words four and five are wasted. DeepSeek's drafter calculates the odds the big model accepts each guess and folds bad bets early, only sending the safe prefix for verification — another ~4–7% speed with no accuracy loss.
6:074. Bounded replay. Long conversations eat storage, and a 2-hour-away follow-up usually forces the server to reread the whole chat. DeepInfer keeps active memory in fast local RAM for ~3 minutes, and because of the architecture, catching up only rereads the last ~140 words — a 1,000-word or 1-million-word prompt resumes essentially instantly.
DeepSeek Recipe
7:22The companion gateway, written in Rust (as is the trend), does two things: it splits prompts along structural boundaries and tokenizes chunks in parallel across all CPU cores (killing the pre-inference lag on huge prompts), and it acts as a prompt-injection shield — malicious system commands get force-encoded as plain text, neutralizing the attack before it reaches the model.
The hardware story
8:11DeepInfer explicitly supports two platforms: Nvidia's new Blackwell architecture and Huawei's domestic Ascend 950 — and pointedly not the previous-generation Hopper/H100. Amid tightening US export restrictions, the message is that a 155-trillion-token operation can run at frontier scale on domestic Chinese silicon, independent of Western hardware if it has to.
The catch
8:50DeepSeek's own docs list what they left out. You get the engine, not the factory: they kept their 100-petabyte storage cluster (10 TB/s), the specialized network routing, and the global load balancer behind closed doors. To actually hit these speeds you still need a serious enterprise setup — at least 128 GPUs running their flagship DeepSeek V4.1 Flash. The video's read on the bigger picture: 9:35for two years the narrative was "whoever buys the most GPUs wins" — DeepSeek keeps showing that smart systems engineering beats brute-force spending, and that puts real pressure on cloud providers marking up open-model hosting.
Claims checked
The repos are real. deepseek-ai/DeepInfer — Apache-2.0, ~21k★, "the high-performance inference framework that powers DeepSeek's large-scale production services… behind DeepSeek V4.1 Flash." deepseek-ai/deepseek-recipe — MIT, ~2.3k★. DeepInfer's own docs state "155 trillion tokens per day… US$38 per GPU-hour (at valley pricing)."
The 155T-token/day and $38/GPU-hour figures are DeepSeek's own production numbers (self-reported, no independent audit); the "~80% gross margin" and "1.8B words/sec" are the video's derived figures, not DeepSeek's official statements. The architectural details (20+20 layer split, 748B/8B/16B MoE, the four engineering moves) are relayed from DeepSeek's release material.
The "smart engineering beats brute force" and the cloud-provider pricing-pressure angles are the creator's commentary, not facts. No sponsor segment.