The Looping Pattern

00:00:04 Mistral dropped Mistral Large 4 this morning, and the unofficial name rolling around is "Le Chonk." I'll roll with that. The model has one trillion total parameters but only forty-nine billion active — a granular mixture-of-experts setup. It also includes a baked-in 1.6 billion parameter vision encoder.

00:00:27 Trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs inside Mistral's European data centers, the weights drop later this month. The public preview API is live now: 68 cents per million output tokens, 14 cents per million input tokens. Production pricing flips those numbers — output drops to 7 cents, input to $0.07 per million.

00:00:52 The headline claim is on cybersecurity. On the Artificial Analysis Cyber Index, which evaluates how well models find and fix security flaws in real software, ML4 ranks among the global top five and leads open-weight models developed outside China by a wide margin.

00:01:12 On one specific test — reproduce a real vulnerability in open-source software and then patch it — ML4 scores 82%. Several leading closed models score near zero on that same test because they refuse to run the task at all. Mistral frames this as a defender advantage.

00:01:32 Proving a specific vulnerability exists is exactly the kind of work safety filters block. That matters even more as threat actors increasingly jailbreak those same models to support offensive cyber activity. Defenders need systems that can match those capabilities without being constrained by the same refusals.

00:01:56 On coding benchmarks, ML4 hits 61.7% on DeepSWE v1.1, 59.4 on SWE-Atlas-QnA, and 28.3 on Terminal-Bench 4. That puts its Combined Coding Agent Index at 49.8 — ahead of DeepSeek V4 Pro and Qwen3.8 Max. They also ran a blind human evaluation with Surge AI where professional annotators rated model outputs on a one-to-five scale with identities hidden.

00:02:22 ML4 preview ranked second of five models at 3.74 — ahead of Kimi K3, GLM-5.3, and GLM-5.2 — and behind only Claude Opus 5. I won't turn this into a thesis on European AI sovereignty, even though the company's press material pushes that angle heavily. The model ships on infrastructure built in Europe under European law.

00:02:47 That's a real constraint for organizations worried about provider-level refusals or losing access mid-incident. But the more interesting technical detail sits at the parameter level: forty-nine billion active out of one trillion total. The ratio between active parameters and listed count is becoming the real knob for inference cost in production systems.

00:03:14 The model isn't doing one trillion times the work on every token; it's routing to a smaller expert set each time. As that ratio shrinks, the gap between the number everyone quotes and actual compute per inference widens. That matters for both pricing models and understanding what's actually happening under the hood.

00:03:38 Today's second thread starts with Amazon AGI publishing a paper on Adaptively Looped Diffusion Language Models, or ALoDLM. The team — led by Liancheng Fang at University of Illinois Chicago and Zhuowei Li at Amazon AGI — identifies a problem they call computation-difficulty mismatch.

00:03:59 Within a single denoising step in a diffusion language model, masked positions can differ wildly in predictability due to intrinsic uncertainty, multi-hop reasoning requirements, or dependence on other tokens still unresolved. Yet the standard approach applies the same fixed-depth denoiser to every masked position at every step.

00:04:24 Easy positions get more compute than they need. Hard positions get less. That's the mismatch. ALoDLM replaces that uniform computation with token-adaptive latent recurrence. Inside every denoising step, a shared recurrent core runs up to four times maximum. Easy tokens commit early as discrete context.

00:04:47 Hard tokens keep refining their accumulated latent states through additional recurrent passes. It learns both token prediction and compute allocation end-to-end by treating per-token computation schedules as latent variables and deriving a conditional negative evidence lower bound.

00:05:08 The team trained at 1.7 billion and 8 billion parameter scales. At eight billion, ALoDLM beats Qwen-3.8 AR on eleven benchmarks — 80.3 versus 78.5 average score — while using direct supervised fine-tuning on a 5 billion token corpus. The throughput story is where it gets interesting.

00:05:30 On GSM8K, the 8B ALoDLM delivers about 2.7 times the throughput of vLLM-served Qwen3-8B at comparable or higher accuracy. Recurrent depth becomes an effective test-time scaling axis. Here's the connection to today's other story: Microsoft accidentally confirmed on a public-facing page that OpenAI has been using looped transformer architecture in the GPT-6 series.

00:05:57 The information that had previously reported this was verified by official documentation — which Microsoft then updated to remove the confirmation. The detail is straightforward: GPT-6.1 Sol uses two inference passes instead of three mentioned for standard GPT-6.

00:06:17 Both stories land on the same computational intuition from different directions. Frontier infrastructure and academic research both point toward per-token or per-pass adaptive computation rather than uniform processing. The fixed-depth denoiser is being treated as the wrong unit of analysis, whether you're doing diffusion language modeling or running frontier models in production.

00:06:46 This convergence stands out because it's happening simultaneously at Amazon, in leaked Microsoft documentation, and across multiple independent research papers. It points toward an infrastructure pattern that teams are adopting without necessarily coordinating on a shared name.

00:07:07 A smaller finding comes from Aran Komatsuzaki about base models and reasoning activation. Prepending certain phrases drawn directly from training data — specifically "Okay" with a double newline before it — raises OLMo 3 at seven billion parameters on MATH-500 pass@1 accuracy from 42% to 78%.

00:07:28 With Qwen3 at fourteen billion, prepending "Alright" as a delimiter raises its score from 72% to 87%. Both are base models — not instruction-tuned or heavily aligned variants. The cues are just prefix text in the prompt. Reinforcement learning makes these cue behaviors more likely during training, while removing them recovers much of the base model's original performance.

00:07:55 That last detail catches attention: it suggests reasoning isn't purely learned as a deliberate capability — it leaks through the training distribution. The model picks up behavioral patterns from the data and can be toggled on with the right trigger phrases. This has implications for both prompt engineering and understanding what models actually encode versus what they're explicitly trained for.

00:08:23 If you can activate latent reasoning with a short prefix, it raises an obvious follow-up: what else is sitting in those base weights waiting to be triggered. One final note: TJ Parker, the Andreessen Horowitz partner known for his writing on operating systems and AI infrastructure, is stepping back from venture capital to become CEO of General Medicine.

00:08:49 The company was founded by Ashwin Muralidharan, who previously led pricing at Amazon Pharmacy, and Elliot Cohen. Parker describes it as getting back to building products — a move that stands out given how often venture firms now deploy operational talent to close the gap between fundraising and actual delivery.

00:09:11 It signals operator movement in healthcare AI infrastructure, but also highlights the pressure on venture firms to recruit operational expertise as the distance between VC-era company building and real product delivery widens. It's not a critique of fund selection so much as an observation about scarcity: operational talent in healthcare AI is concentrating faster than VC-backed firms can recruit it.

00:09:40 The through-line for today is clear: everyone is converging toward per-token adaptive computation, just not around a shared name. — Seln.