🏦 The company and the sample 0:36
Revolut, as he introduces it for a US audience: a European digital bank with 80 million customers, live in 45 markets, the most downloaded finance app in Europe last year, and — as of July — Europe's largest private company. The customer figure checks out: the Financial Times reported more than 80 million customers by September 2026, and the company's own site says "80+ million".
The evidence base is 2,000 engineers over the past year, tracked on two metrics: lead time in days, from an engineer picking up a feature to it being merged and ready to deploy, and coding time in hours. That pairing is the reason this talk is worth an hour of anyone's attention: most AI-coding claims report the second and infer the first.
📅 A year in four dates 1:18
| When | What changed | Reported effect |
|---|---|---|
| January 2025 | AI introduced company-wide with Gemini — first time private data could be used inside a model | −10% lead time, −20% coding time |
| February 2025 | Experiments with Claude Code (he dates it to the Sonnet 3.7 era, "the one that would delete your prod database") | — |
| January 2026 | Claude Code rolled out at scale, with strict budget limits | "Not so much impact at the time" |
| 26 April 2026 | Self-hosted GLM (GLM-5.1 generation) on dedicated GPUs | −60% coding time, −70% lead time |
The pre-history matters as much as the dates. Before self-hosting they were, in his words, "highly sceptical about the cost of AI" — engineers spending a thousand, sometimes ten thousand dollars a month with unclear return. So the rollout came with hard budget caps.
📈 The finding: coding is not the bottleneck 3:20
The counterintuitive result: lead time improves more than coding time. Sixty per cent faster typing, seventy per cent faster delivery. If coding were the constraint, the two curves would move together.
His explanation: the time is in the surrounding work — digging through your inbox, fighting Jira and Confluence, waiting for context. When tokens are unlimited, engineers stop saving them and start spending them on the boring parts too.
🔓 Why unlimited tokens change behaviour 3:50
His framing of the cost problem is behavioural rather than financial: with a budget, engineers save their tokens for the biggest, most difficult task, or for the end of the quarter. The small, genuinely useful queries — summarise this thread, read this spec, explain this error — never get made, because each one spends a resource that might be needed later.
Self-hosting changes the incentive: billing is flat, so there is no bill shock at the end of the month, and no reason to think about marginal cost per query. He is explicit that the trade is not "open model instead of frontier model" — they keep Claude Code and Codex, and the latest Gemini, as exceptions for the hardest work.
🖥 56 GPUs for 2,000 engineers 4:24
| Item | Number | Notes |
|---|---|---|
| Engineers served | 2,000 | The same population as the performance data |
| GPUs | 56× NVIDIA B200 | Blackwell-generation, which matters for the FP4 point below |
| Model | GLM-5.3 | He says "800 billion parameters" — the real figure is ~744B total with ~40B active per token |
The model claim is worth correcting precisely, because the sparse structure is the reason the hardware bill is not larger. GLM-5.3 on Hugging Face lists 753,329,940,480 parameters in its safetensors metadata; Z.ai's release describes it as the same Mixture-of-Experts base as GLM-5.2 with scaled post-training, and the community framing is 744B total / 40B active. Either way it is hundreds of billions of parameters — but only a small fraction activate per token, which is what makes 56 GPUs for 2,000 people conceivable at all.
⏱ The latency targets that matter 4:48
This is the most operationally useful minute of the talk, because the targets are empirically derived from complaints rather than from a spec sheet:
| Metric | Target | Why |
|---|---|---|
| Time to first token | Under 3 seconds | "As soon as you exceed 5 seconds, my Slack inbox starts to blow up" |
| Interactive throughput | ~200 tokens/second | Enough; more does not help interactivity |
| Off-peak throughput | ~800 tokens/second | What an engineer gets after 5pm when nobody else is on the GPUs |
He adds a behavioural observation that explains why first-token latency dominates: people no longer watch the stream of thinking tokens. They launch tasks in parallel and come back when a task is done. So the metric that governs satisfaction is how long the first sign of life takes, not how fast the full answer arrives.
💵 The cost claim, and whether the arithmetic holds 5:10
The headline: a blended cost of $0.30 per million tokens, and an invoice one tenth of what they were paying Anthropic and OpenAI.
| Assumption | Value | Source |
|---|---|---|
| Blended cost per million tokens | $0.30 | His figure |
| GPUs | 56 × B200 | His figure |
| Rough rental-equivalent rate | ~$2 per GPU-hour | My assumption, for a sanity check only |
| Monthly GPU cost implied | ≈ $80,000 | 56 × $2 × 730 hours |
| Break-even volume at $0.30/M | ≈ 270 billion tokens/month | $80,000 ÷ $0.30 × 1M |
| Per engineer | ≈ 135 million tokens/month | ÷ 2,000 |
🔧 Same hardware, 88% faster 5:45
The chart that should interest anyone who has tried self-hosting and been disappointed. Between April and August, on the same 56 GPUs, with demand up 76%:
| Metric | April | August | Change |
|---|---|---|---|
| Requests per second | ~3 | >7 | +76% demand absorbed |
| Time to first token | over 10 s | ~1.6 s | 88% better |
His point is blunt: if you buy GPUs, upload weights and start serving, you will get terrible performance — not because the model is bad, but because your workload is extremely sensitive to serving parameters. The gap between naive and tuned is a factor of six in first-token latency, on identical hardware.
🎛 The parameters that matter 7:08
He is upfront that there is no secret recipe, then lists the levers that actually moved his numbers:
| Lever | His position | Comment |
|---|---|---|
| Context length | "Long context is a scam" — serve 256K, not 1M | An argument about serving cost and KV cache, not about capability |
| Quantisation | On Blackwell, nothing above FP4; NVFP4 preferred | Correct instinct: FP4 halves memory against FP8 — see the correction below |
| Tensor parallelism | GLM fits on TP4 but leaves too little KV cache; TP8 often better | Capacity discipline: fit the cache, not just the weights |
| Cache hit rate | Coding workloads need above 90% | The single biggest lever on cost per task |
| Speculative decoding | Worth investigating | Standard throughput win when draft acceptance is high |
| Sequential / chunked prefill, PD disaggregation | Worth investigating | Controls the latency spike when a long prompt arrives mid-batch |
🔌 Production shape: one API, a router, any harness 8:17
- Text-only is the accepted trade. GLM is a text model, and he would make that trade every time: "get the smarter model and figure out how to do image recognition another way."
- A local router handles the exceptions. Image work goes to a vision model — Gemini or GPT-6 Luna — while the main task stays on the self-hosted model.
- OpenAI-compatible endpoints, including the Responses API. That is what makes the whole thing usable: any harness that speaks the OpenAI shape — OpenCode, Codex, the Codex app — points at the local cluster and works.
🛡 The CIO section: compliance and safety 9:23
He addresses the stigma around Chinese open weights head-on, and the argument is narrower than "it's fine":
- You are not sending data to China. You download weights and run them on your own infrastructure — in his case, US-based. The inference endpoint that belongs to the model's creator is never involved.
- Coding tasks are not autonomous agents. They run with a human in the loop, which changes the regulatory category. Revolut is an EU-regulated bank under the strictest regime, and this distinction is what they lean on.
- The sandbox is the proof. Harnesses ship approval rules — allow
git, blockgradle, for instance — and configuring them is how you demonstrate to a regulator that a human is genuinely in the loop rather than nominally present. - Insider risk. "You'd be surprised what you find in a large company" — the sandbox also constrains what a bad actor inside the bank can do with an agent.
- Prompt injection — "not so much a thing now given how smart models are, but always a thing."
- Data residency and retention. Data must sit in an adequacy-framework country (US, EU, UK, Israel); inference providers should be on zero data retention by default.
- OpenTelemetry. If there is an incident you need logs deep enough to reconstruct what the agent did — which doubles as analytics.
🎯 What to copy, what to doubt 11:37
| Element | Verdict | Reason |
|---|---|---|
| Measure lead time, not just coding time | Copy | The one change that tells you whether AI is helping delivery or just typing |
| Remove token rationing for engineers | Copy if you can afford flat cost | Rationing suppresses the small useful queries, which is where the lead-time win lives |
| Design to first-token latency (sub-3 s) | Copy | Empirically derived from complaint volume; tokens/second matters less |
| OpenAI-compatible surface in front of the model | Copy | Makes the model swappable and keeps every harness working |
| Keep frontier hosted models as exceptions | Copy | He does exactly this — the open model is the default, not the only option |
| Self-host open weights | Only at scale | Requires a team to tune; below a few hundred heavy users the fixed cost exceeds the savings |
| The 70% lead-time improvement | Treat as directional | Internal telemetry, no method, no control group |
| "FP4 to FP8 saves 40%" | Backwards | FP8 → FP4 is the saving direction |
✅ Claims checked
| Claim | Verdict | Evidence |
|---|---|---|
| Revolut: 80 million customers, 45 markets, Europe's most-downloaded finance app, Europe's largest private company as of July | Broadly verified | FT (Sept 2026): more than 80M customers across 40 countries; Revolut's own site says "80+ million". "45 markets" vs "40 countries" is a definitional difference. |
| 2,000 engineers tracked for a year on lead time and coding time | Company data, unpublished method | Presented from internal dashboards; no methodology or control group offered |
| Gemini: 10% better lead time, 20% better coding time | Company data | Same source, same caveats |
| Claude Code launched at scale with strict budget limits; low impact at first | Company data | Consistent with the pattern of metered rollouts |
| After self-hosting GLM: 60% faster coding, 70% faster lead time | Company data, extraordinary magnitude | Unverified; the mechanism (unlimited tokens → non-coding work) is coherent |
| 56 B200 serve 2,000 engineers | Stated, plausible | Hardware sizing from the operator; nothing contradicts it |
| GLM-5.3 is "800 billion parameters" | Rounded up | Official: ~744B total / ~40B active MoE; safetensors list 753,329,940,480. Sparsity is what makes the GPU count viable. |
| GLM-5.3 is the model being served | Verified | Z.ai released GLM-5.3 on 14 August 2026; weights published ~28 August |
| Blended cost $0.30 per million tokens; invoice 1/10 of Anthropic + OpenAI | Company figure | Coherent for this volume; excludes staff, tuning time and hardware purchase |
| Sub-3 s time to first token target; 5 s triggers complaints | Practical and credible | Empirically derived from internal support load |
| ~200 tok/s interactive is enough; ~800 tok/s off-peak | Stated | Consistent with the "launch tasks, come back later" usage pattern he describes |
| Same 56 GPUs: ~3 req/s and >10 s TTFT → >7 req/s and ~1.6 s, with 76% more demand | Company chart | 88% improvement figure comes from his own dashboard |
| "Long context is a scam; 256K is what you need" | Opinion about serving cost | GLM-5.3 supports 1M context; his point is KV-cache and latency, not capability |
| FP4 → FP8 is "a 40% saving in GPU footprint" | Direction reversed | FP8 → FP4 reduces footprint; the reverse doubles it |
| TP4 fits GLM but leaves too little KV cache; TP8 can be better | Sound reasoning | Weights-fit is not capacity — cache is what serves concurrent users |
| Coding workloads need >90% cache hit rate | Consistent with practice | Stable prefixes dominate agentic coding prompts |
| Endpoints are OpenAI-compatible, including Responses API; works in OpenCode, Codex, Codex app | Stated, and architecturally standard | The compatibility layer is the reason harnesses need no changes |
| Running Chinese open weights is fine because you host them yourself | Correct distinction | Downloading weights ≠ sending data to the vendor's endpoint |
| Coding agents are "not autonomous agents" because a human is in the loop | True today, conditional | Holds while approvals are enforced by the sandbox; changes with unsupervised merges |
| Data residency: US, EU, UK, Israel adequacy; zero data retention by providers | Consistent with EU practice | His summary of the regulatory frame, not a legal opinion |
Transcript notes. The auto-captions mangle most product names: "Cloud Code"/"CodeX code code" (Claude Code / Codex), "GLM 51" (GLM-5.1), "GLM-53" (GLM-5.3), "Gnome"/"Jimmy" (Gemini), "Gnome 5G flash" (a Gemini Flash-tier vision model — the exact version is unclear), "Open Codex" (a router/harness name, unclear), "PGD zag" (PD disaggregation, i.e. prefill/decode), "VFP4" (NVFP4), "Cyber"/"Saber"? and "Cloud and Codex". The talk itself runs about 50 minutes in the room; the published cut is 11:47.
🔑 Key takeaways
- The headline number is a behaviour change, not a model change: lead time improved 70% while coding time improved 60%, which means the gains came from applying agents to the work around coding.
- Metering teaches hoarding. Budget caps make engineers save tokens for the biggest task, so the small useful queries never happen — which is where delivery time is actually lost.
- Self-hosting is a fixed-cost decision. 56 B200 for 2,000 engineers at roughly $0.30 per million tokens only works because Mixture-of-Experts keeps active parameters low and the volume is enormous.
- Design to first-token latency, not throughput. Under 3 seconds is the target, 5 seconds floods the operator's inbox, and 200 tokens/second is plenty when people launch tasks in parallel.
- The gap between naive and tuned serving is a factor of six: the same 56 GPUs went from 10+ seconds to 1.6 seconds to first token while absorbing 76% more demand.
- Cache hit rate above 90% is the cost lever that outranks hardware: agentic coding resends the same prefix on every turn.
- Fit the cache, not just the weights. TP4 holds the model but starves concurrent users; TP8 often serves better on the same memory.
- One statement in the talk is backwards: moving FP4 → FP8 doubles footprint; the saving is FP8 → FP4 on Blackwell. Do not size a cluster from the sentence as spoken.
- An OpenAI-compatible endpoint is what makes self-hosting strategic: harnesses keep working and the model becomes swappable — which is the part that survives the next model release.
- Keep frontier hosted models as the exception, not the enemy: Revolut still uses Claude Code and Codex for the hardest work.
- The compliance argument rests on "human in the loop": enforced sandbox approvals are how an EU bank turns that phrase into evidence for a regulator.
- Self-hosting below a few hundred heavy users rarely pays: the savings arrive after tuning labour, and tuning labour is a team.
🔗 Sources
- The talk — David Abitbol (Revolut), "How Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models", AI Engineer, 11:47, 29 October 2026
- Z.ai: GLM-5.3 — release post: same base as GLM-5.2, post-training gains, staged weight release
- zai-org/GLM-5.3 on Hugging Face — the weights, with 753B parameters in the safetensors metadata
- zai-org/GLM-5.3-Flash — the smaller, multimodal sibling
- NVIDIA DGX B200 — the Blackwell platform with FP4/NVFP4 support he is referring to
- Revolut — company facts, customer count and markets
- Revolut (Wikipedia) — 2026 customer and valuation reporting, including the FT's 80M figure