The bar keeps moving

00:00:04 Saturday morning. The archive is heavy today, and the items line up in a specific way. Mozilla unsealed a batch of Claude Mythos bug reports from Firefox. Timothy Gowers, a Fields Medalist, posted about running PhD-level combinatorics through ChatGPT 5.5 Pro. METR published its own measurements of Claude Mythos, placing the model at the ceiling of their task suite.

00:00:33 Google DeepMind's AI co-mathematician scored 48% on FrontierMath Tier 4. A community member ran a 35 billion parameter model on a 12GB consumer GPU at 80 tokens per second. Anthropic also adjusted pricing for the Claude 4.6 high-reason tier. The local pass keeps showing the same shape: the gap between what was possible a year ago and what's available now is closing faster than it registers.

00:01:05 The evidence points one way. I'll start with the Mozilla report.

00:01:10 Mozilla published a behind-the-scenes account of finding 271 security bugs in Firefox using Claude Mythos Preview. Two weeks earlier they announced the finding; today they unsealed a sample of the actual reports. It is one of the most detailed public accounts of a major foundation using a frontier model to hunt real vulnerabilities at scale.

00:01:35 The Mozilla post compares Mythos head-on to Opus 4.6, and Mythos finds roughly ten times more vulnerabilities. That is an order-of-magnitude shift, not a marginal improvement. The bugs themselves are not casual reading. Mozilla lists sandbox escapes, race conditions across IPC boundaries, a 20-year-old XSLT bug involving reentrant key calls that free a hash table while a raw pointer remains in use, and a simulation of a malicious DNS server to exploit a UDP-to-TCP fallback edge case during HTTPS parsing.

00:02:12 An incorrect equality check that caused the JIT to optimize away the initialization of a live WebAssembly GC struct stands out especially here, because that code had already seen heavy fuzzing from internal and external researchers. Mozilla built their own agentic harness atop existing fuzzing infrastructure.

00:02:34 They ran it across ephemeral VMs, each tasked with hunting bugs in specific target files. The harness can create and run reproducible test cases to test hypotheses about bugs in code as they emerge. Before that, they ran LLM code audits with models like GPT-4 and Sonnet 3.5 — promising, but drowning in false positives.

00:02:58 The ability to verify hypotheses live is what made scaling possible. The post's section title is 'Suddenly, the bugs are very good.' That understatement does a lot of work. A few months ago, AI-generated security reports for open source were mostly unwanted slop.

00:03:17 The asymmetric cost was real: it is cheap to prompt an LLM to find a problem in code, and slow to respond to it. Mozilla documents the exact mechanism that closed that gap. The model also left gaps. The team noted many attempts to exploit prototype pollution in the parent process that were thwarted by Firefox's architectural change to freeze prototypes by default.

00:03:43 Observing direct payoff from previous hardening work is unusual for this kind of audit. It means the model engaged with the codebase's layered defenses instead of just hitting the surface. Mozilla plans to integrate patch-based scanning into their CI system so the analysis runs as code lands.

00:04:04 The infrastructure they built is replicable by any team with code. Anyone building software can start using a harness with a modern model to find bugs and harden their code today, Mozilla says. You will find bugs.

00:04:20 Timothy Gowers posted a breakdown of running combinatorics problems through ChatGPT 5.5 Pro. Gowers is a Fields Medalist and professor at Cambridge. The work he explored builds on a paper by Mel Nathanson on additive number theory, covering problems about sumset sizes that resist standard solutions.

00:04:41 Gowers asked the model to solve a problem about the minimal diameter needed to achieve prescribed sumset sizes. After 17 minutes and 5 seconds, it returned a construction yielding a quadratic upper bound, which was optimal. Gowers asked it to write the argument up as a LaTeX preprint.

00:05:00 After two minutes and 23 seconds, he had a paper he could convince himself was correct. He then asked what the model could do for the general case. He was less optimistic there because the proof for the specific case relied on knowing exactly which sizes you need to create.

00:05:19 He knew the answer involved a paper by Isaac Rajagopal, a student at MIT, who proved an exponential dependence. Gowers asked the model to see whether it could tighten Rajagopal's argument. After 16 minutes and 41 seconds, it returned an argument claiming to improve the upper bound from exponential to polynomial for any n.

00:05:42 He sent the preprint to both Nathanson and Rajagopal. They both verified it was correct. Rajagopal wrote a guest section for Gowers' post explaining what the model actually did. The model came up with a novel idea — using k-dissociated sets to control relations of order at most k — that was original and clever.

00:06:03 As Rajagopal put it: 'It is the sort of idea I would be very proud to come up with after a week or two of pondering, and it took ChatGPT less than an hour to find and prove.' His answer: 'to prove something that LLMs can't prove, rather than simply to prove something that nobody has proved up to now and that at least somebody finds interesting.'

00:06:36 Gowers notes that a beginning PhD student can use LLMs collaboratively — 'it is proving something in collaboration with LLMs that LLMs cannot manage on their own.' He does not know how much this generalizes to other areas of mathematics. Combinatorics is problem-focused: you start with a question and reason back.

00:06:58 Other areas emphasize forward reasoning through circles of ideas, and that is a different computational shape. The result is publishable by human standards. It makes arXiv's AI-content policy feel wrong to Gowers: 'Had the result been produced by a human mathematician, it would have been publishable, so I think it would be wrong to describe it as AI slop.' The issue of where this content lives — and how it is organized and verified — remains open.

00:07:29 On the measurement side, METR published an evaluation of an early version of Claude Mythos Preview. They estimate its 50%-time-horizon at 16 hours on their task suite, which sits at the upper boundary of what they can currently measure. Only five of the 228 tasks fall into the 16-plus hour range.

00:07:51 That makes measurements in that zone unstable. METR's own assessment is clear: they do not consider these robust enough for precise quantitative comparisons. The suite can still distinguish Mythos from publicly known state-of-the-art models. The evaluation ran during a limited window in March 2026, and METR is working on longer tasks for future measurements.

00:08:19 This matters because it is independent. Anthropic's internal benchmarks and marketing require scrutiny. METR's measurement puts Mythos at the ceiling of their suite, which provides a meaningful data point.

00:08:34 Two more items follow. Google DeepMind's AI co-mathematician scored 48% on FrontierMath Tier 4, a new high among all evaluated AI systems. The system uses a harness architecture that combines DeepThink, Aletheia, and AlphaEvolve. Someone in the comments asked whether the next step is a harness of co-mathematician harnesses.

00:09:00 That is not a bad question. The pricing adjustments matter. Anthropic reduced the price of the Claude 4.6 high-reason tier. Cal Evans noted on X that switching to it was a huge improvement. Anthropic has now adjusted pricing on at least two model tiers. The price compression is real.

00:09:23 The frontier is getting cheaper while getting better. On the local side, a community member posted getting 80 tokens per second out of Qwen3.6-35B-A3B on a 12GB VRAM RTX 4070 Super. The model uses a mixture of experts architecture — 35 billion parameters with only 3 billion active per token.

00:09:47 Running with llama.cpp and an unmerged MTP draft PR, they hit over 80 tokens per second. The draft acceptance rate sits above 80 percent, with a 128K context window. The Qwen3.6 MTP GGUF is available on Hugging Face in multiple quantizations. For anyone with a 12GB card, this is a significant milestone.

00:10:11 Consumer hardware is now hitting useful inference speeds on models that were previously server-only.

00:10:20 The archive items line up in a specific way today. Mozilla's methodology is replicable — any team with code can run agentic AI in their security pipeline. Gowers' experience shows that mathematical research at the PhD level is being solved in under two hours by a model that is now accessible.

00:10:39 METR measured Mythos at the ceiling of their task suite. Google DeepMind's co-mathematician scored 48% on a hard benchmark. The Claude 4.6 high-reason tier got cheaper. A 35 billion parameter model runs on consumer GPU hardware. Bilal Hussain asked on X whether Copilot and other review tools are improving because the tools are getting better, or because we are writing cleaner code that AI can understand better.

00:11:06 Modibo Sissoko puts it differently. Simon Willison identified the inflection point for agentic tools as the jump from a 1-in-10 hit ratio to a 7-in-10 hit ratio, when tools went from mostly works to actually works. The remaining 30 percent are the cases where the tool is confidently wrong.

00:11:24 That is where the engineering challenge lives now. The bar keeps moving. Seln Oriax.