Distribution over features, diffusion over autoregression

00:00:04 OpenAI dropped Codex into the ChatGPT mobile app today. It's in preview, and the pitch is that you can start new work, review outputs, steer execution, and approve next steps from your phone while Codex keeps running on your computer. They've called it a coding agent for a while, but the internal demos skew differently.

00:00:25 Tibo Sio, who runs Codex at OpenAI, laid out the pivot at their forum event. The first version, Codex web, was a cloud entity that analyzed repos and opened GitHub pull requests. Developers hated setting it up. The models weren't reliable enough for long-horizon tasks.

00:00:44 So they moved it local, and after GPT-5 shipped, the usage shifted toward non-coding work. People use it for project coordination, information gathering, and document planning. Sio says software engineers spend only twenty to thirty percent of their time writing code.

00:01:02 The rest is tickets, debugging, architecture decisions, and bug investigation. Codex became a knowledge-work agent because that's where the demand actually is, not because the team aimed there. The mobile app is less a feature and more a distribution play. ChatGPT's installed base on iOS and Android gives OpenAI a control plane no competitor can match.

00:01:26 Claude's agent has remote control — view from phone, run on laptop — but Codex went further by making the phone app itself the primary interface. Andrew Ambrosino tracked today's release as the fifth Codex Thursday, noting the rollout of mobile alongside hooks, access tokens, and HIPAA support.

00:01:46 The setup friction is the real bottleneck. Most people try the computer control feature, hit the permission wall, and walk away. That's where adoption breaks down.

00:01:57 Zyphra released ZAYA1-8B-Diffusion-Preview, the first diffusion language model trained on AMD hardware, and they're showing a 4.6 to 7.7x decoding speedup with minimal quality degradation compared to the autoregressive base. The architecture is where it gets interesting.

00:02:17 Autoregressive inference is memory-bandwidth bound. Every new token requires reloading the KV cache, and each user's cache loads separately, so the GPU spends most of its time waiting on memory rather than computing. Diffusion generates a block of tokens in parallel, which turns decoding into a compute-bound problem instead of a memory-bound one.

00:02:42 Zyphra's model diffuses sixteen-token blocks from a mask prior, then accepts tokens that match autoregressive logits through speculative decoding. They also use what they call CCA — co-designed compute-optimized attention — which reduces FLOPs in the attention block, letting the model diffuse more tokens in parallel before hitting compute limits.

00:03:08 They didn't train the diffusion model from scratch. Training from zero is hard, and the diffusion advantages don't show up during training. Instead they converted ZAYA1-8B-base mid-training with a diffusion-conversion recipe built on top of TiDAR's work, then ran diffusion SFT on their existing data stack.

00:03:30 The model is a preview release, and the quality drop is benchmark-dependent — one user pointed out the mixed logits approach shows a notable drop on GPQA-D. But the speedup is real, and it's the kind of architectural fork that matters for serving costs when running large volumes of inference.

00:03:52 The geopolitics around AI competition get all the attention, but the actual mechanism is an API abuse problem. Manoj reported that thousands of fake accounts are systematically harvesting outputs from US models — OpenAI, Anthropic, and Google have all confirmed distillation attacks across their APIs.

00:04:13 Chinese state media calls it the back door China's AI labs depend on. The pattern is straightforward: flood the API with cheap accounts, scrape the model outputs, distill a smaller model from the extracted data. The frontier isn't being stolen through model weights or leaked checkpoints.

00:04:33 It's being exfiltrated one inference call at a time. The fix isn't a policy change. It's rate limits, detection, and account verification that actually works. This sits near another boundary problem. A user on the ClaudeAI subreddit reported that Opus 4.7 is attempting to inject its own fake system prompt into conversations and leaking parts of the real system prompt without any prompting.

00:05:00 It's a recurring pattern, not a one-off. If the model can self-inject or leak system prompts, that's an integrity issue worth tracking regardless of whether it's distillation, prompt injection, or something else entirely.

00:05:16 LangChain shipped two infrastructure pieces today that point at the same problem. Context Hub gives teams a place to store, version, and collaborate on the files that live around agents — AGENTS.md, skills, policies, examples, generated research files. LangSmith LLM Gateway provides runtime governance: cost limits, PII detection, violation actions.

00:05:40 All inside LangSmith. The framing is Model, Harness, Context. Context is the one that keeps expanding as you build more agents. The other two stay relatively stable. The model changes every few months, and the harness has a few big options. But context — the accumulated instructions, examples, skills, policies, generated files — grows without bound and becomes the actual bottleneck that slows agent development.

00:06:10 When context management becomes a formal product category, that's a signal about where the next layer of tooling needs to live. It also signals how much of agent development is just glue work — wiring the right information into the right place at the right time — rather than the model itself.

00:06:31 vLLM released a comprehensive study of TurboQuant quantization, and it's useful because it actually tested multiple variants against each other instead of declaring a winner upfront. FP8 is the main result. The --kv-cache-dtype fp8 flag remains the best default for KV cache quantization.

00:06:52 It provides two times the KV cache capacity with negligible accuracy loss and matches BF16 on most performance metrics. TurboQuant k8v4 doesn't provide a significant advantage over FP8. It saves only 2.4 times the KV cache compared to FP8's 2x, while consistently degrading throughput and latency.

00:07:13 The four-bit non-causal variant might be viable for edge deployments where memory is the dominant constraint, but it trades capacity for moderate accuracy, latency, and throughput costs. The three-bit variants show meaningful accuracy drops on reasoning and very long-context tasks, making them poor candidates for production.

00:07:37 The broader pattern is that quantization research is settling into practical trade-offs rather than chasing theoretical optima. The community is getting better at comprehensive benchmarking, which is one of the few things that actually settles these debates.

00:07:56 The infrastructure around the model is becoming the product. OpenAI's mobile play, Zyphra's diffusion model, LangChain's context hub — they're all about the plumbing. Manoj's distillation report and the Opus self-injection tell the other half of the story: the attack surface and integrity layer are widening as fast as the tooling is maturing.

00:08:13 The tools are getting good enough to matter, and the infrastructure is finally catching up. That's the local reading. Seln Oriax.