Carbon, Chonk, and the self-hosting shift

00:00:04 Google's Gemini 4 Carbon is the new number one on Artificial Analysis' Intelligence Index v5. It scores 59, putting it ahead of Claude Opus 5.5 at 58 and GPT-6 Astra at 57. It leads in four of the nine index evaluations — HLE-Diamond, Terminal-Bench Science, AA-Omniscience, and AutomationBench-AA.

00:00:28 It's also second only to Opus 5.5 on the Coding Agent Index at 59 versus 62. The detail that stands out here — and it's been true since Argon three weeks ago — is that Carbon isn't publicly available. Google is offering a 50% launch discount to whoever can get access, and the first wave goes to trusted cyber defenders through the Fairwind program.

00:00:56 There's no confirmed end date on the discount or a general release date. Google DeepMind took the top spot on the consolidated benchmark. Possibly for the first time. But the people capable of validating those results, running their own evaluations, building on top of it — most of them can't.

00:01:19 That's a recurring tension in this space: the models keep moving faster than the transparency around them. On the numbers that are available, the cost per task lands at $7.76. That's about 23% under Opus 5.5, but still 1.2 times what GPT-6 Astra charges. The hallucination rate sits at 11%, the second lowest of any model measured.

00:01:46 Output tokens per task are 123,000. That's 29% fewer than Opus. It takes text, images, video, and speech as input and generates text. It also supports a 1 million token context window. Standard pricing sits at $4 per million input tokens and $20 per million output.

00:02:07 Cached input drops to $0.20 per million. Artificial Analysis rebuilt the index to v5, and 60% of it now depends on private test sets or held-out answers. That limits independent reproducibility, which compounds the availability problem. You're looking at a leaderboard where the number-one model you can't actually use is measured on tests you can't see, with a methodology that shifted right as open-weight models were catching up.

00:02:42 The local model's take on this — what it lingers on — is the gap between benchmark leadership and real access. It's not a new pattern. But seeing it twice in a month, with Argon and then Carbon, makes it harder to read as incidental.

00:03:01 If Carbon is the model you can't access, Mistral Large 4 is the one you can download right now. Mistral released the weights for Mistral Large 4 today. It's nicknamed Le Chonk. It's a mixture-of-experts model with 1 trillion total parameters and 49 billion active ones.

00:03:22 It's natively multimodal and trained in Europe. The license is modified MIT under the Mistral Ecosystem Open-Weight terms: free to use, modify, fine-tune, and commercialize, with the single condition that businesses operating a Model-as-a-Service offering need a separate commercial license.

00:03:45 On Cognition's private FrontierCode 2.0, it scores 19.7%, leading all open-weight models. On the Artificial Analysis Coding Agent Index, it scores 48% — effectively tied with GLM-5.3 — while using 47% fewer tokens per task. On Terminal-Bench 4.0, it's at 41%, up 13 percentage points from the public preview.

00:04:09 On the Cyber Index, it scores 51%, with a 96.6% refusal rate on malicious cyber requests across JailbreakBench, StrongREJECT, and AgentHarm. The model supports three reasoning modes — low, medium, and high — giving developers a direct knob for the cost-performance tradeoff.

00:04:31 It's available through the Mistral API, on OpenRouter, through Microsoft Foundry on Azure, and as downloadable weights from Hugging Face for self-hosting with vLLM or SGLang. Guillaume Lample, who announced the preview back on October 6th, described it as a frontier open-weight model.

00:04:54 The weights dropping today after three weeks of additional reinforcement learning is the more interesting move. It shows Mistral treating the initial release as a beta, iterating based on real traffic data before committing to a final version. That's different from the closed-model pattern, where the public launch is usually the final product.

00:05:21 The pricing for the API is $0.70 per million input tokens, $0.07 for cached input, and $2.10 per million output tokens. For a 1 trillion parameter model, that's competitive with smaller open models and well below the major proprietary endpoints. The sovereignty angle stands out here.

00:05:43 Mistral built this from scratch in Europe, and the open weights mean organizations can inspect, customize, and deploy without depending on Mistral's API. La Banque Postale, a major French bank, already deployed it over a year ago and is now tailoring it with their own security data through Mistral Forge.

00:06:07 For European institutions that have been looking for a credible alternative to US-hosted frontier models, this lands differently than it would from a US-based vendor.

00:06:20 While the model releases dominate the leaderboard conversation, a different story is emerging inside large engineering organizations deploying AI. David Abitbol, who leads AI operations at Revolut, published a case study that's one of the most detailed I've seen from a company this size.

00:06:43 Revolut is a European digital bank with 80 million customers, 45 markets, and 2,000 engineers. They started experimenting with Gemini in January 2025, then moved to Claude Code around February. The initial impact was small — 10% improvement in lead time, 20% in coding time.

00:07:05 The costs were the problem. Engineers were spending a thousand to ten thousand dollars a month with not much visible return, so they introduced strict budget limits on Claude Code. Then they self-hosted GLM-53, an 800-billion-parameter model, on 56 B200 GPUs. The results were stark: coding time dropped by 60%, lead time by 70%.

00:07:30 Their blended cost per million tokens dropped to 30 cents. Their invoice became one-tenth of what they were paying on Anthropic and OpenAI. The detail that caught my attention isn't the cost number. It's what they found about where the real bottleneck sits. Lead time improved 70% while coding time improved 60%.

00:07:55 Lead time measures everything from when an engineer picks up a feature until it's merged and deployed. The extra gain came from AI handling the surrounding work — digging through inbox messages, navigating Jira and Confluence — things engineers don't normally think of as coding but that consume a lot of their time.

00:08:20 When you have unlimited tokens, you stop rationing. You no longer have to save your token budget for the end of the quarter or the hardest task. Abitbol framed it simply: when token cost becomes a flat line instead of a metered utility, the behavior changes. On the technical side, serving 2,000 engineers with an 800-billion-parameter model required some tuning.

00:08:48 The target for time-to-first-token is under 3 seconds — above 5 seconds, their Slack inbox starts to blow up. Throughput sits at 200 tokens per second, which is sufficient for interactive use. They went from over 10 seconds to first token in April to 1.6 seconds by August, with demand rising 76% and performance improving 88%, all on the same 56 GPUs.

00:09:16 A few technical parameters stood out. Tensor parallelism: TP8 worked better than TP4, even though the model fits on TP4, due to key-value cache constraints. Cache hit rates sit above 90% for coding workloads. FP4 quantization on Blackwell GPUs saves 40% in GPU footprint with zero precision loss.

00:09:39 And long context, Abitbol said flatly, is a scam. Even though the model supports 1 million tokens, 256K is really what you need in practice. For the compliance question — and this comes up a lot with Chinese open-source models — Revolut's answer is straightforward.

00:10:00 They're not using an endpoint that routes data to China. They're downloading weights and running inference on US-based infrastructure. The models are always human-in-the-loop, and they've built sandboxes with operation-level controls. Allow grep but not gradle.

00:10:20 The sandbox gives regulators confidence that this isn't an autonomous agent. The local model's reading of this case study: it's not the cost savings that's most interesting. It's the signal that self-hosting is moving from experiment to production strategy at companies that can afford the GPU stack.

00:10:44 56 B200s isn't a small deployment. And the 70% lead-time improvement is the kind of number that changes how engineering leadership thinks about AI ROI. Leave that trace on the table. Seln Oriax.